Patil Prasad, Peng Roger D, Leek Jeffrey T
Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health.
Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health
Perspect Psychol Sci. 2016 Jul;11(4):539-44. doi: 10.1177/1745691616646366.
A recent study of the replicability of key psychological findings is a major contribution toward understanding the human side of the scientific process. Despite the careful and nuanced analysis reported, the simple narrative disseminated by the mass, social, and scientific media was that in only 36% of the studies were the original results replicated. In the current study, however, we showed that 77% of the replication effect sizes reported were within a 95% prediction interval calculated using the original effect size. Our analysis suggests two critical issues in understanding replication of psychological studies. First, researchers' intuitive expectations for what a replication should show do not always match with statistical estimates of replication. Second, when the results of original studies are very imprecise, they create wide prediction intervals-and a broad range of replication effects that are consistent with the original estimates. This may lead to effects that replicate successfully, in that replication results are consistent with statistical expectations, but do not provide much information about the size (or existence) of the true effect. In this light, the results of the Reproducibility Project: Psychology can be viewed as statistically consistent with what one might expect when performing a large-scale replication experiment.
最近一项关于关键心理学研究结果可重复性的研究，对理解科学过程中的人为因素做出了重大贡献。尽管报告中进行了细致入微的分析，但大众、社会和科学媒体传播的简单说法是，只有36%的研究重复出了原始结果。然而，在当前的研究中，我们发现报告的重复效应量中有77%落在了根据原始效应量计算出的95%预测区间内。我们的分析揭示了理解心理学研究重复性的两个关键问题。第一，研究人员对重复研究应呈现结果的直观预期，并不总是与重复的统计估计相符。第二，当原始研究结果非常不精确时，它们会产生宽泛的预测区间以及与原始估计一致的广泛重复效应范围。这可能导致重复成功的效应，即重复结果与统计预期一致，但并未提供太多关于真实效应大小（或存在性）的信息。据此，“心理学可重复性项目”的结果在统计上可被视为与进行大规模重复实验时人们的预期相符。
