Heyard Rachel, Pawel Samuel, Frese Joris, Voelkl Bernhard, Würbel Hanno, McCann Sarah, Held Leonhard, Wever Kimberley E, Hartmann Helena, Townsin Louise, Zellers Stephanie
Center for Reproducible Science, University of Zurich Institute of Epidemiology Biostatistics and Prevention, Zurich, Switzerland.
Department of Biostatistics, University of Zurich Institute of Epidemiology Biostatistics and Prevention, Zurich, Switzerland.
R Soc Open Sci. 2025 Jul 15;12(7):242076. doi: 10.1098/rsos.242076. eCollection 2025 Jul.
Reproducibility is recognized as essential to scientific progress and integrity. Replication studies and large-scale replication projects, aiming to quantify different aspects of reproducibility, have become more common. Since no standardized approach to measuring reproducibility exists, a diverse set of metrics has emerged and a comprehensive overview is needed. We conducted a scoping review to identify large-scale replication projects that used metrics and methodological papers that proposed or discussed metrics. The project list was compiled by the authors. For the methodological papers, we searched Scopus, MedLine, PsycINFO and EconLit. Records were screened in duplicate against pre-defined inclusion criteria. Demographic information on included records and information on reproducibility metrics used, suggested or discussed was extracted. We identified 49 large-scale projects and 97 methodological papers and extracted 50 metrics. The metrics were characterized based on type (formulas and/or statistical models, frameworks, graphical representations, studies and questionnaires, algorithms), input required and appropriate application scenarios. Each metric addresses a distinct question. Our review provides a comprehensive resource in the form of a 'live', interactive table for future replication teams and meta-researchers, offering support in how to select the most appropriate metrics that are aligned with research questions and project goals.
可重复性被认为是科学进步和诚信的关键。旨在量化可重复性不同方面的重复研究和大规模重复项目已变得更加普遍。由于不存在衡量可重复性的标准化方法，因此出现了各种各样的指标，需要进行全面概述。我们进行了一项范围审查，以识别使用指标的大规模重复项目以及提出或讨论指标的方法学论文。项目清单由作者编制。对于方法学论文，我们检索了Scopus、MedLine、PsycINFO和EconLit。根据预先定义的纳入标准对记录进行了重复筛选。提取了纳入记录的人口统计学信息以及所使用、建议或讨论的可重复性指标的信息。我们识别出49个大规模项目和97篇方法学论文，并提取了50个指标。根据类型（公式和/或统计模型、框架、图形表示、研究和问卷、算法）、所需输入和适当的应用场景对这些指标进行了表征。每个指标都解决一个独特的问题。我们的审查以一个“实时”交互式表格的形式为未来的重复研究团队和元研究人员提供了全面的资源，为如何选择与研究问题和项目目标相一致的最合适指标提供支持。
