Division of Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women's Hospital, Boston, MA, USA.
Department of Medicine, Harvard Medical School, Boston, MA, USA.
Nat Commun. 2022 Aug 31;13(1):5126. doi: 10.1038/s41467-022-32310-3.
Studies that generate real-world evidence on the effects of medical products through analysis of digital data collected in clinical practice provide key insights for regulators, payers, and other healthcare decision-makers. Ensuring reproducibility of such findings is fundamental to effective evidence-based decision-making. We reproduce results for 150 studies published in peer-reviewed journals using the same healthcare databases as original investigators and evaluate the completeness of reporting for 250. Original and reproduction effect sizes were positively correlated (Pearson's correlation = 0.85), a strong relationship with some room for improvement. The median and interquartile range for the relative magnitude of effect (e.g., hazard ratio/hazard ratio) is 1.0 [0.9, 1.1], range [0.3, 2.1]. While the majority of results are closely reproduced, a subset are not. The latter can be explained by incomplete reporting and updated data. Greater methodological transparency aligned with new guidance may further improve reproducibility and validity assessment, thus facilitating evidence-based decision-making. Study registration number: EUPAS19636.
通过对临床实践中收集的数字数据进行分析，生成关于医疗产品效果的真实世界证据的研究为监管机构、支付方和其他医疗保健决策者提供了关键见解。确保此类发现的可重复性对于基于有效证据的决策至关重要。我们使用与原始研究人员相同的医疗保健数据库，对在同行评审期刊上发表的 150 项研究的结果进行了复制，并对 250 项研究的报告完整性进行了评估。原始和复制的效果大小呈正相关（Pearson 相关系数=0.85），这种关系非常强，但仍有一些改进的空间。效果（例如，风险比/风险比）的中位数和四分位距为 1.0 [0.9, 1.1]，范围为 [0.3, 2.1]。虽然大多数结果都得到了很好的复制，但有一部分没有。后者可以通过不完整的报告和更新的数据来解释。与新指南保持一致的更大方法透明度可能会进一步提高可重复性和有效性评估，从而促进基于证据的决策。研究注册编号：EUPAS19636。
