Kellogg School of Management, Northwestern University
Kellogg School of Management, Northwestern University.
Perspect Psychol Sci. 2016 Sep;11(5):730-749. doi: 10.1177/1745691616662243.
We review and evaluate selection methods, a prominent class of techniques first proposed by Hedges (1984) that assess and adjust for publication bias in meta-analysis, via an extensive simulation study. Our simulation covers both restrictive settings as well as more realistic settings and proceeds across multiple metrics that assess different aspects of model performance. This evaluation is timely in light of two recently proposed approaches, the so-called p-curve and p-uniform approaches, that can be viewed as alternative implementations of the original Hedges selection method approach. We find that the p-curve and p-uniform approaches perform reasonably well but not as well as the original Hedges approach in the restrictive setting for which all three were designed. We also find they perform poorly in more realistic settings, whereas variants of the Hedges approach perform well. We conclude by urging caution in the application of selection methods: Given the idealistic model assumptions underlying selection methods and the sensitivity of population average effect size estimates to them, we advocate that selection methods should be used less for obtaining a single estimate that purports to adjust for publication bias ex post and more for sensitivity analysis-that is, exploring the range of estimates that result from assuming different forms of and severity of publication bias.
我们通过广泛的模拟研究，回顾和评估了选择方法，这是一类由 Hedges（1984 年）首次提出的技术，用于评估和调整荟萃分析中的发表偏倚。我们的模拟涵盖了限制条件和更现实的条件，并使用了多个评估模型性能不同方面的指标。鉴于最近提出的两种方法，即所谓的 p 曲线和 p 均匀方法，可以将它们视为原始 Hedges 选择方法的替代实现，因此这次评估是及时的。我们发现，p 曲线和 p 均匀方法的表现相当不错，但在为三者设计的限制条件下，表现不如原始 Hedges 方法。我们还发现它们在更现实的条件下表现不佳，而 Hedges 方法的变体表现良好。最后，我们告诫要谨慎使用选择方法：鉴于选择方法所基于的理想化模型假设以及它们对总体平均效应大小估计的敏感性，我们主张选择方法不应用于获得单一的估计值，该估计值声称可以事后调整发表偏倚，而应更多地用于敏感性分析，即探索在假设不同形式和严重程度的发表偏倚的情况下得出的估计值范围。
