Devezer Berna, Navarro Danielle J, Vandekerckhove Joachim, Ozge Buzbas Erkan
Department of Business, University of Idaho, Moscow, Idaho, USA.
School of Psychology, University of New South Wales, Sydney, Australia.
R Soc Open Sci. 2021 Mar 31;8(3):200805. doi: 10.1098/rsos.200805.
Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific practices in experimental sciences. However, most methodological reform attempts suffer from similar mistakes and over-generalizations to the ones they aim to address. We argue that this can be attributed in part to lack of formalism and first principles. Considering the costs of allowing false claims to become canonized, we argue for formal statistical rigor and scientific nuance in methodological reform. To attain this rigor and nuance, we propose a five-step formal approach for solving methodological problems. To illustrate the use and benefits of such formalism, we present a formal statistical analysis of three popular claims in the metascientific literature: (i) that reproducibility is the cornerstone of science; (ii) that data must not be used twice in any analysis; and (iii) that exploratory projects imply poor statistical practice. We show how our formal approach can inform and shape debates about such methodological claims.
当前科学领域的方法论改革尝试，是对实验科学中方法论和科学实践普遍缺乏严谨性的回应。然而，大多数方法论改革尝试都犯了与它们旨在解决的问题类似的错误，并存在过度概括的情况。我们认为，这在一定程度上可归因于缺乏形式化和第一原理。考虑到让错误论断被奉为圭臬的代价，我们主张在方法论改革中要有形式化的统计严谨性和科学的细微差别。为实现这种严谨性和细微差别，我们提出了一种解决方法论问题的五步形式化方法。为说明这种形式化的用途和益处，我们对元科学文献中的三个流行论断进行了形式化统计分析：（i）可重复性是科学的基石；（ii）在任何分析中数据都不能被重复使用；（iii）探索性项目意味着糟糕的统计实践。我们展示了我们的形式化方法如何为有关此类方法论论断的辩论提供信息并塑造辩论。
