Feinberg School of Medicine, Northwestern University.
Multivariate Behav Res. 2023 May-Jun;58(3):598-615. doi: 10.1080/00273171.2022.2066500. Epub 2022 Jun 4.
A prominent approach to studying the replication crisis has been to conduct replications of several different scientific findings as part of the same research effort. The reported proportion of findings that these programs determined failed to replicate have become important statistics in the replication crisis. However, these "failure rates" are based on decisions about whether individual studies replicated, which are themselves subject to statistical uncertainty. In this article, we examine how that uncertainty impacts the accuracy of reported failure rates and find that the reported failure rates can be substantially biased and highly variable. Indeed, very high or very low failure rates could arise from chance alone.
一种研究复制危机的突出方法是在同一研究工作中对几个不同的科学发现进行复制。这些计划确定未能复制的发现比例已成为复制危机中的重要统计数据。然而，这些“失败率”是基于对个别研究是否复制的决定，而这些决定本身就存在统计不确定性。在本文中，我们研究了这种不确定性如何影响报告的失败率的准确性，发现报告的失败率可能会产生实质性的偏差和高度的可变性。事实上，非常高或非常低的失败率可能仅仅是由于偶然因素造成的。
