Faculty of Psychology and Educational Sciences, KU Leuven, Leuven, Belgium.
Department of Psychology, University of Tuebingen, Tuebingen, Germany.
PLoS One. 2023 Apr 13;18(4):e0284243. doi: 10.1371/journal.pone.0284243. eCollection 2023.
Sharing research data allows the scientific community to verify and build upon published work. However, data sharing is not common practice yet. The reasons for not sharing data are myriad: Some are practical, others are more fear-related. One particular fear is that a reanalysis may expose errors. For this explanation, it would be interesting to know whether authors that do not share data genuinely made more errors than authors who do share data. (Wicherts, Bakker and Molenaar 2011) examined errors that can be discovered based on the published manuscript only, because it is impossible to reanalyze unavailable data. They found a higher prevalence of such errors in papers for which the data were not shared. However, (Nuijten et al. 2017) did not find support for this finding in three large studies. To shed more light on this relation, we conducted a replication of the study by (Wicherts et al. 2011). Our study consisted of two parts. In the first part, we reproduced the analyses from (Wicherts et al. 2011) to verify the results, and we carried out several alternative analytical approaches to evaluate the robustness of the results against other analytical decisions. In the second part, we used a unique and larger data set that originated from (Vanpaemel et al. 2015) on data sharing upon request for reanalysis, to replicate the findings in (Wicherts et al. 2011). We applied statcheck for the detection of consistency errors in all included papers and manually corrected false positives. Finally, we again assessed the robustness of the replication results against other analytical decisions. Everything taken together, we found no robust empirical evidence for the claim that not sharing research data for reanalysis is associated with consistency errors.
分享研究数据可以让科学界验证和扩展已发表的工作。然而，数据共享并不是常见的做法。不共享数据的原因有很多：有些是实际的，有些则更多是出于恐惧。一个特别的担忧是重新分析可能会暴露错误。对于这个解释，了解那些不共享数据的作者是否真的比那些共享数据的作者犯了更多的错误会很有趣。（Wicherts、Bakker 和 Molenaar 2011）研究了仅基于已发表的手稿就可以发现的错误，因为无法重新分析不可用的数据。他们发现，在那些不共享数据的论文中，这种错误的发生率更高。然而，（Nuijten 等人，2017）在三项大型研究中并没有发现这一发现的支持。为了更清楚地了解这种关系，我们对（Wicherts 等人，2011）的研究进行了复制。我们的研究包括两个部分。在第一部分中，我们复制了（Wicherts 等人，2011）的分析，以验证结果，我们还进行了几种替代的分析方法，以评估结果对其他分析决策的稳健性。在第二部分，我们使用了一个独特的更大的数据集，该数据集源自（Vanpaemel 等人，2015），请求对数据进行重新分析以进行数据共享，以复制（Wicherts 等人，2011）的发现。我们使用 statcheck 检测所有纳入论文中的一致性错误，并手动纠正假阳性。最后，我们再次评估了复制结果对其他分析决策的稳健性。总的来说，我们没有发现确凿的经验证据表明，不重新分析研究数据与一致性错误有关。
