Boekel Wouter, Wagenmakers Eric-Jan, Belay Luam, Verhagen Josine, Brown Scott, Forstmann Birte U
University of Amsterdam, Amsterdam, The Netherlands.
University of Amsterdam, Amsterdam, The Netherlands.
Cortex. 2015 May;66:115-33. doi: 10.1016/j.cortex.2014.11.019. Epub 2015 Jan 14.
A recent 'crisis of confidence' has emerged in the empirical sciences. Several studies have suggested that questionable research practices (QRPs) such as optional stopping and selective publication may be relatively widespread. These QRPs can result in a high proportion of false-positive findings, decreasing the reliability and replicability of research output. A potential solution is to register experiments prior to data acquisition and analysis. In this study we attempted to replicate studies that relate brain structure to behavior and cognition. These structural brain-behavior (SBB) correlations occasionally receive much attention in science and in the media. Given the impact of these studies, it is important to investigate their replicability. Here, we attempt to replicate five SBB correlation studies comprising a total of 17 effects. To prevent the impact of QRPs we employed a preregistered, purely confirmatory replication approach. For all but one of the 17 findings under scrutiny, confirmatory Bayesian hypothesis tests indicated evidence in favor of the null hypothesis ranging from anecdotal (Bayes factor < 3) to strong (Bayes factor > 10). In several studies, effect size estimates were substantially lower than in the original studies. To our knowledge, this is the first multi-study confirmatory replication of SBB correlations. With this study, we hope to encourage other researchers to undertake similar replication attempts.
实证科学领域最近出现了一场“信心危机”。多项研究表明，诸如选择性停止和选择性发表等有问题的研究行为（QRPs）可能相对普遍。这些有问题的研究行为可能导致高比例的假阳性结果，降低研究成果的可靠性和可重复性。一个潜在的解决方案是在数据采集和分析之前对实验进行注册。在本研究中，我们试图复制将脑结构与行为和认知相关联的研究。这些脑结构 - 行为（SBB）相关性研究偶尔会在科学界和媒体上受到广泛关注。鉴于这些研究的影响力，调查其可重复性很重要。在此，我们试图复制五项SBB相关性研究，共计17个效应。为防止有问题的研究行为的影响，我们采用了预先注册的、纯粹验证性的复制方法。对于所审查的17项结果中除一项之外的所有结果，验证性贝叶斯假设检验表明有证据支持零假设，证据强度从轶事性的（贝叶斯因子<3）到强的（贝叶斯因子>10）不等。在几项研究中，效应大小估计值明显低于原始研究。据我们所知，这是首次对SBB相关性进行的多研究验证性复制。通过这项研究，我们希望鼓励其他研究人员进行类似的复制尝试。
