Department of Computer Science, Princeton University, Princeton, NJ 08544, USA.
Center for Information Technology Policy, Princeton University, Princeton, NJ 08544, USA.
Sci Adv. 2024 May 3;10(18):eadk3452. doi: 10.1126/sciadv.adk3452. Epub 2024 May 1.
Machine learning (ML) methods are proliferating in scientific research. However, the adoption of these methods has been accompanied by failures of validity, reproducibility, and generalizability. These failures can hinder scientific progress, lead to false consensus around invalid claims, and undermine the credibility of ML-based science. ML methods are often applied and fail in similar ways across disciplines. Motivated by this observation, our goal is to provide clear recommendations for conducting and reporting ML-based science. Drawing from an extensive review of past literature, we present the REFORMS checklist (recommendations for machine-learning-based science). It consists of 32 questions and a paired set of guidelines. REFORMS was developed on the basis of a consensus of 19 researchers across computer science, data science, mathematics, social sciences, and biomedical sciences. REFORMS can serve as a resource for researchers when designing and implementing a study, for referees when reviewing papers, and for journals when enforcing standards for transparency and reproducibility.
机器学习 (ML) 方法在科学研究中迅速普及。然而，这些方法的采用伴随着有效性、可重复性和通用性的失败。这些失败可能会阻碍科学进步，导致无效主张周围形成虚假共识，并破坏基于机器学习的科学的可信度。ML 方法在不同学科中经常以类似的方式应用和失败。受此观察的启发，我们的目标是为基于机器学习的科学提供明确的建议。我们从对过去文献的广泛回顾中，提出了基于 ML 的科学的 REFORMS 清单 (REcommendations for machine-learning-based science)。它由 32 个问题和一对指导方针组成。REFORMS 是基于来自计算机科学、数据科学、数学、社会科学和生物医学科学的 19 名研究人员的共识而开发的。REFORMS 可以作为研究人员在设计和实施研究时、评审人员在评审论文时以及期刊在执行透明度和可重复性标准时的资源。
