Ioannidis John P A, Trikalinos Thomas A, Zintzaras Elias
Clinical Trials and Evidence-based Medicine Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece.
J Clin Epidemiol. 2006 Oct;59(10):1023-32. doi: 10.1016/j.jclinepi.2006.02.013. Epub 2006 Aug 7.
Meta-analyses are routinely evaluated for the presence of large between-study heterogeneity. We examined whether it is also important to probe whether there is extreme between-study homogeneity.
We used heterogeneity tests with left-sided statistical significance for inference and developed a Monte Carlo simulation test for testing extreme homogeneity in risk ratios across studies, using the empiric distribution of the summary risk ratio and heterogeneity statistic. A left-sided P=0.01 threshold was set for claiming extreme homogeneity to minimize type I error.
Among 11,803 meta-analyses with binary contrasts from the Cochrane Library, 143 (1.21%) had left-sided P-value <0.01 for the asymptotic Q statistic and 1,004 (8.50%) had left-sided P-value <0.10. The frequency of extreme between-study homogeneity did not depend on the number of studies in the meta-analyses. We identified examples where extreme between-study homogeneity (left-sided P-value <0.01) could result from various possibilities beyond chance. These included inappropriate statistical inference (asymptotic vs. Monte Carlo), use of a specific effect metric, correlated data or stratification using strong predictors of outcome, and biases and potential fraud.
Extreme between-study homogeneity may provide useful insights about a meta-analysis and its constituent studies.
系统评价通常会针对研究间的巨大异质性进行评估。我们研究了探究研究间是否存在极端同质性是否也很重要。
我们使用具有左侧统计显著性的异质性检验进行推断，并使用汇总风险比和异质性统计量的经验分布，开发了一种蒙特卡罗模拟检验，用于检验各研究风险比的极端同质性。设定左侧P = 0.01的阈值来判定极端同质性，以尽量减少I型错误。
在Cochrane图书馆中11803项采用二元对比的系统评价中，143项（1.21%）的渐近Q统计量左侧P值<0.01，1004项（8.50%）的左侧P值<0.10。研究间极端同质性的频率并不取决于系统评价中的研究数量。我们发现了一些例子，其中研究间的极端同质性（左侧P值<0.01）可能是由多种非偶然因素导致的。这些因素包括不恰当的统计推断（渐近法与蒙特卡罗法）、使用特定的效应指标、相关数据或使用结果的强预测因子进行分层，以及偏倚和潜在欺诈。
研究间的极端同质性可能会为系统评价及其组成研究提供有用的见解。
