School of Public Health and Preventive Medicine, Monash University, 553 St. Kilda Road, Melbourne, Victoria, 3004, Australia.
Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada.
Syst Rev. 2021 Apr 16;10(1):112. doi: 10.1186/s13643-021-01670-0.
Investigations of transparency, reproducibility and replicability in science have been directed largely at individual studies. It is just as critical to explore these issues in syntheses of studies, such as systematic reviews, given their influence on decision-making and future research. We aim to explore various aspects relating to the transparency, reproducibility and replicability of several components of systematic reviews with meta-analysis of the effects of health, social, behavioural and educational interventions.
The REPRISE (REProducibility and Replicability In Syntheses of Evidence) project consists of four studies. We will evaluate the completeness of reporting and sharing of review data, analytic code and other materials in a random sample of 300 systematic reviews of interventions published in 2020 (Study 1). We will survey authors of systematic reviews to explore their views on sharing review data, analytic code and other materials and their understanding of and opinions about replication of systematic reviews (Study 2). We will then evaluate the extent of variation in results when we (a) independently reproduce meta-analyses using the same computational steps and analytic code (if available) as used in the original review (Study 3), and (b) crowdsource teams of systematic reviewers to independently replicate a subset of methods (searches for studies, selection of studies for inclusion, collection of outcome data, and synthesis of results) in a sample of the original reviews; 30 reviews will be replicated by 1 team each and 2 reviews will be replicated by 15 teams (Study 4).
The REPRISE project takes a systematic approach to determine how reliable systematic reviews of interventions are. We anticipate that results of the REPRISE project will inform strategies to improve the conduct and reporting of future systematic reviews.
对科学的透明度、可重复性和可复制性的调查主要针对单个研究。鉴于系统评价对决策和未来研究的影响，在研究综合中探索这些问题同样至关重要，例如系统评价。我们旨在探索与健康、社会、行为和教育干预措施的效果进行系统评价的几个方面的透明度、可重复性和可复制性。
REPRISE（证据综合中的再现性和可复制性）项目由四项研究组成。我们将评估 2020 年发表的 300 篇干预措施系统评价的随机样本中报告和共享审查数据、分析代码和其他材料的完整性（研究 1）。我们将对系统评价的作者进行调查，以探讨他们对共享审查数据、分析代码和其他材料的看法，以及他们对系统评价复制的理解和意见（研究 2）。然后，我们将评估以下两种情况的结果差异：（a）我们使用与原始审查中相同的计算步骤和分析代码（如果可用）独立复制荟萃分析（研究 3）；以及（b）众包系统评价团队独立复制原始审查中的一个方法子集（研究检索、纳入研究选择、结果数据收集和结果综合）；每个团队将复制 30 篇评论，15 个团队将复制 2 篇评论（研究 4）。
REPRISE 项目采用系统方法来确定干预措施的系统评价的可靠性。我们预计，REPRISE 项目的结果将为改进未来系统评价的实施和报告提供信息。
