Department of Child Neurology, UMC Utrecht Brain Center, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.
Epileptic Disord. 2023 Jun;25(3):285-296. doi: 10.1002/epd2.20010. Epub 2023 May 24.
The quest for epilepsy biomarkers is on the rise. Variables with statistically significant group-level differences are often misinterpreted as biomarkers with sufficient discriminative power. This study aimed to demonstrate the relationship between significant group-level differences and a variable's power to discriminate between individuals.
We simulated normal-distributed datasets from hypothetical populations with varying sample sizes (25-800), effect sizes (Cohen's d: .25-2.50), and variability (standard deviation: 10-35) to assess the impact of these parameters on significance and discriminative power. The simulation data were illustrated by assessing the discriminative power of a potential real-case biomarker-the EEG beta band power-to diagnose generalized epilepsy, using data from 66 children with generalized epilepsy and 385 controls. Additionally, we evaluated recently reported epilepsy biomarkers by comparing their effect sizes to our simulation-derived effect size criterion.
Group size affects significance but not discriminative power. Discriminative power is much more related to variability and effect size. Our real data example supported these simulation results by demonstrating that group-level significance does not translate, one to one, into discriminative power. Although we found a significant difference in the beta band power between children with and without epilepsy, the discriminative power was poor due to a small effect size. A Cohen's d of at least 1.25 is required to reach good discriminative power in univariable prediction modeling. Slightly over 60% of the biomarkers in our literature search met this criterion.
Rather than statistical significance of group-level differences, effect size should be used as an indicator of a variable's biomarker potential. The minimal required effects size for individual biomarkers-a Cohen's d of 1.25-is large. This calls for multivariable approaches, in which combining multiple variables with smaller effect sizes could increase the overall effect size and discriminative power.
寻找癫痫生物标志物的研究方兴未艾。具有统计学显著组间差异的变量常被错误地解释为具有足够判别能力的生物标志物。本研究旨在展示变量的组间差异显著性与其个体间判别能力之间的关系。
我们模拟了具有不同样本量（25-800）、效应大小（Cohen's d：.25-2.50）和变异性（标准差：10-35）的假设人群的正态分布数据集，以评估这些参数对显著性和判别力的影响。通过评估 EEG β波段功率作为诊断全面性癫痫的潜在实际生物标志物的判别力，利用 66 例全面性癫痫患儿和 385 例对照者的数据来说明模拟数据。此外，我们还通过将其效应大小与我们的模拟衍生效应大小标准进行比较，评估了最近报道的癫痫生物标志物。
组间大小影响显著性但不影响判别力。判别力与变异性和效应大小关系更为密切。我们的真实数据示例通过证明组间显著性不能一一转化为判别力，支持了这些模拟结果。虽然我们发现癫痫患儿和无癫痫患儿之间的β波段功率存在显著差异，但由于效应量较小，判别力较差。在单变量预测建模中，需要达到至少 1.25 的 Cohen's d 才能获得良好的判别力。我们文献检索中略多于 60%的生物标志物符合此标准。
不是组间差异的统计学显著性，而是效应大小应该作为变量生物标志物潜力的指标。个体生物标志物的最小所需效应大小 - Cohen's d 为 1.25 - 较大。这需要多变量方法，其中将具有较小效应大小的多个变量结合起来可以增加总体效应大小和判别力。
