Department of Psychology, University of Western Ontario.
Department of Human Development and Family Sciences, University of Texas at Austin.
J Pers Soc Psychol. 2017 Aug;113(2):230-243. doi: 10.1037/pspi0000049.
Several researchers recently outlined unacknowledged costs of open science practices, arguing these costs may outweigh benefits and stifle discovery of novel findings. We scrutinize these researchers' (a) statistical concern that heightened stringency with respect to false-positives will increase false-negatives and (b) metascientific concern that larger samples and executing direct replications engender opportunity costs that will decrease the rate of making novel discoveries. We argue their statistical concern is unwarranted given open science proponents recommend such practices to reduce the inflated Type I error rate from .35 down to .05 and simultaneously call for high-powered research to reduce the inflated Type II error rate. Regarding their metaconcern, we demonstrate that incurring some costs is required to increase the rate (and frequency) of making true discoveries because distinguishing true from false hypotheses requires a low Type I error rate, high statistical power, and independent direct replications. We also examine pragmatic concerns raised regarding adopting open science practices for relationship science (preregistration, open materials, open data, direct replications, sample size); while acknowledging these concerns, we argue they are overstated given available solutions. We conclude benefits of open science practices outweigh costs for both individual researchers and the collective field in the long run, but that short term costs may exist for researchers because of the currently dysfunctional academic incentive structure. Our analysis implies our field's incentive structure needs to change whereby better alignment exists between researcher's career interests and the field's cumulative progress. We delineate recent proposals aimed at such incentive structure realignment. (PsycINFO Database Record
几位研究人员最近概述了开放科学实践中未被承认的成本，认为这些成本可能超过收益，并阻碍新发现的发现。我们仔细研究了这些研究人员的（a）对错误阳性率提高会导致错误阴性率增加的统计关注，以及（b）对更大样本和执行直接复制会产生机会成本从而降低新发现率的元科学关注。我们认为，鉴于开放科学的支持者建议采取这些做法将第一类错误率从 0.35 降低到 0.05，并同时呼吁进行高影响力的研究以降低第二类错误率，因此他们的统计关注是没有根据的。关于他们的元关注，我们表明，为了提高真实发现的速度（和频率），需要承担一些成本，因为区分真实和虚假假设需要低的第一类错误率、高的统计功效和独立的直接复制。我们还考察了在关系科学中采用开放科学实践所引发的实用问题（预先登记、开放材料、开放数据、直接复制、样本量）；尽管承认这些关注，但我们认为，鉴于现有解决方案，这些关注被夸大了。我们的结论是，从长远来看，开放科学实践对个人研究人员和整个领域都有好处，但由于目前学术激励结构的功能失调，研究人员短期内可能会付出代价。我们的分析意味着，我们领域的激励结构需要改变，以便在研究人员的职业利益和领域的累积进展之间更好地保持一致。我们阐述了最近旨在调整这种激励结构的提案。
