Buzbas Erkan O, Devezer Berna, Baumgaertner Bert
Department of Mathematics and Statistical Science, University of Idaho, Moscow, ID 83844, USA.
Department of Business, University of Idaho, Moscow, ID 83844, USA.
R Soc Open Sci. 2023 Mar 15;10(3):221042. doi: 10.1098/rsos.221042. eCollection 2023 Mar.
The scientific reform movement has proposed openness as a potential remedy to the putative reproducibility or replication crisis. However, the conceptual relationship among openness, replication experiments and results reproducibility has been obscure. We analyse the logical structure of experiments, define the mathematical notion of idealized experiment and use this notion to advance a theory of reproducibility. Idealized experiments clearly delineate the concepts of replication and results reproducibility, and capture key differences with precision, allowing us to study the relationship among them. We show how results reproducibility varies as a function of the elements of an idealized experiment, the true data-generating mechanism, and the closeness of the replication experiment to an original experiment. We clarify how openness of experiments is related to designing informative replication experiments and to obtaining reproducible results. With formal backing and evidence, we argue that the current 'crisis' reflects inadequate attention to a theoretical understanding of results reproducibility.
科学改革运动提出开放性作为应对假定的可重复性或复制危机的一种潜在补救措施。然而,开放性、复制实验和结果可重复性之间的概念关系一直模糊不清。我们分析实验的逻辑结构,定义理想化实验的数学概念,并利用这一概念推进可重复性理论。理想化实验清晰地界定了复制和结果可重复性的概念,并精确地捕捉到关键差异,使我们能够研究它们之间的关系。我们展示了结果可重复性如何作为理想化实验的要素、真实数据生成机制以及复制实验与原始实验的接近程度的函数而变化。我们阐明了实验的开放性如何与设计信息丰富的复制实验以及获得可重复的结果相关联。有了形式上的支持和证据,我们认为当前的“危机”反映出对结果可重复性的理论理解关注不足。