School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran.
Department of Pediatrics, University of Tennessee Health Science Center, Memphis, TN, USA.
Brain Topogr. 2019 Sep;32(5):882-896. doi: 10.1007/s10548-019-00718-8. Epub 2019 May 25.
Statistical significance testing is a necessary step in connectivity analysis. Several statistical test methods have been employed to assess the significance of functional connectivity, but the performance of these methods has not been thoroughly evaluated. In addition, the effects of the intrinsic brain connectivity and background couplings on performance of statistical test methods in task-based studies have not been investigated yet. The background couplings may exist independent of cognitive state and can be observed on both pre- and post-stimulus time intervals. The background couplings may be falsely detected by a statistical test as task-related connections, which can mislead interpretations of the task-related functional networks. The aim of this study was to investigate the relative performance of four commonly used non-parametric statistical test methods-surrogate, demeaned surrogate, bootstrap resampling, and Monte Carlo permutation methods-in the presence of background couplings and noise, with different signal-to-noise ratios (SNRs). Using simulated electrocorticographic (ECoG) datasets and phase locking value (PLV) as a measure of functional connectivity, we evaluated the performances of the statistical test methods utilizing sensitivity, specificity, accuracy, and receiver operating curve (ROC) analysis. Furthermore, we calculated optimal p values for each statistical test method using the ROC analysis, and found that the optimal p values were increased by decreasing the SNR. We also found that the optimal p value of the bootstrap resampling was greater than that of other methods. Our results from the simulation datasets and a real ECoG dataset, as an illustrative case report, revealed that the bootstrap resampling is the most efficient non-parametric statistical test for identifying the significant PLV of ECoG data, especially in the presence of background couplings.
统计显著性检验是连接分析的必要步骤。已经采用了几种统计检验方法来评估功能连接的显著性，但这些方法的性能尚未得到彻底评估。此外，内在脑连接和背景耦合对基于任务的研究中统计检验方法性能的影响尚未得到研究。背景耦合可能独立于认知状态存在，并可以在前刺激和后刺激时间间隔上观察到。背景耦合可能被统计检验错误地检测为与任务相关的连接，这可能会导致对与任务相关的功能网络的解释产生误导。本研究的目的是研究在存在背景耦合和噪声的情况下，四种常用的非参数统计检验方法（替代、去均值替代、自举重采样和蒙特卡罗置换方法）的相对性能，以及不同的信噪比（SNR）。使用模拟脑电（ECoG）数据集和相位锁定值（PLV）作为功能连接的度量，我们利用灵敏度、特异性、准确性和接收器操作曲线（ROC）分析评估了统计检验方法的性能。此外，我们使用 ROC 分析为每种统计检验方法计算了最优的 p 值，发现随着 SNR 的降低，最优的 p 值增加。我们还发现，自举重采样的最优 p 值大于其他方法。我们从模拟数据集和一个真实的 ECoG 数据集的结果，作为一个说明性案例报告，表明自举重采样是识别 ECoG 数据显著 PLV 的最有效非参数统计检验方法，特别是在存在背景耦合的情况下。
