Centre for Primary Care, NIHR School for Primary Care Research, Institute of Population Health, University of Manchester, Manchester, United Kingdom.
PLoS One. 2013 Jul 26;8(7):e69930. doi: 10.1371/journal.pone.0069930. Print 2013.
Heterogeneity has a key role in meta-analysis methods and can greatly affect conclusions. However, true levels of heterogeneity are unknown and often researchers assume homogeneity. We aim to: a) investigate the prevalence of unobserved heterogeneity and the validity of the assumption of homogeneity; b) assess the performance of various meta-analysis methods; c) apply the findings to published meta-analyses.
We accessed 57,397 meta-analyses, available in the Cochrane Library in August 2012. Using simulated data we assessed the performance of various meta-analysis methods in different scenarios. The prevalence of a zero heterogeneity estimate in the simulated scenarios was compared with that in the Cochrane data, to estimate the degree of unobserved heterogeneity in the latter. We re-analysed all meta-analyses using all methods and assessed the sensitivity of the statistical conclusions. Levels of unobserved heterogeneity in the Cochrane data appeared to be high, especially for small meta-analyses. A bootstrapped version of the DerSimonian-Laird approach performed best in both detecting heterogeneity and in returning more accurate overall effect estimates. Re-analysing all meta-analyses with this new method we found that in cases where heterogeneity had originally been detected but ignored, 17-20% of the statistical conclusions changed. Rates were much lower where the original analysis did not detect heterogeneity or took it into account, between 1% and 3%.
When evidence for heterogeneity is lacking, standard practice is to assume homogeneity and apply a simpler fixed-effect meta-analysis. We find that assuming homogeneity often results in a misleading analysis, since heterogeneity is very likely present but undetected. Our new method represents a small improvement but the problem largely remains, especially for very small meta-analyses. One solution is to test the sensitivity of the meta-analysis conclusions to assumed moderate and large degrees of heterogeneity. Equally, whenever heterogeneity is detected, it should not be ignored.
异质性在荟萃分析方法中起着关键作用，并且会极大地影响结论。但是，未知的真实异质性水平，并且研究人员通常假定为同质性。我们的目的是：a）研究未观察到的异质性的流行程度和同质性假设的有效性；b）评估各种荟萃分析方法的性能；c）将研究结果应用于已发表的荟萃分析。
我们查阅了 2012 年 8 月 Cochrane 图书馆中可用的 57,397 项荟萃分析。使用模拟数据，我们评估了各种荟萃分析方法在不同情况下的性能。在模拟情况下，零异质性估计的出现频率与 Cochrane 数据中的出现频率进行了比较，以估计后者中未观察到的异质性程度。我们使用所有方法重新分析了所有荟萃分析，并评估了统计结论的敏感性。Cochrane 数据中的未观察到的异质性水平似乎很高，尤其是对于小型荟萃分析。在检测异质性和返回更准确的总体效应估计方面，戴西蒙尼-劳德（DerSimonian-Laird）方法的自举版本表现最佳。使用此新方法重新分析所有荟萃分析后，我们发现，在最初检测到但忽略了异质性的情况下，17-20％的统计结论发生了变化。在原始分析未检测到异质性或考虑到异质性的情况下，这种情况要低得多，为 1％至 3％。
当缺乏异质性证据时，标准做法是假定同质性并应用更简单的固定效应荟萃分析。我们发现，假定同质性通常会导致误导性分析，因为很可能存在但未检测到异质性。我们的新方法仅略有改进，但问题仍然存在，尤其是对于非常小的荟萃分析。一种解决方案是测试荟萃分析结论对假定的中等和高度异质性的敏感性。同样，一旦检测到异质性，就不应忽略它。
