Wayant Cole, Page Matthew J, Vassar Matt
Department of Biomedical Sciences, Oklahoma State University Center for Health Sciences, Tulsa.
School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia.
JAMA Oncol. 2019 Nov 1;5(11):1550-1555. doi: 10.1001/jamaoncol.2019.2564.
IMPORTANCE: Reproducible research practices are essential to biomedical research because these practices promote trustworthy evidence. In systematic reviews and meta-analyses, reproducible research practices ensure that summary effects used to guide patient care are stable and trustworthy. OBJECTIVE: To evaluate the reproducibility in theory of meta-analyses in oncology systematic reviews cited by the 49 National Comprehensive Cancer Network (NCCN) guidelines for the treatment of cancer by site and evaluate whether Cochrane reviews or systematic reviews that report adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines use more reproducible research practices. DESIGN, SETTING, AND PARTICIPANTS: A cross-sectional investigation of all systematic reviews with at least 1 meta-analysis and at least 1 included randomized clinical trial (RCT) that are cited by NCCN guidelines for treatment of cancer by site. We scanned the reference list of all NCCN guidelines (n = 49) for potential systematic reviews and meta-analyses. All retrieved studies were screened, and data were extracted, independently and in duplicate. The analysis was carried out between May 6, 2018, and January 28, 2019. MAIN OUTCOMES AND MEASURES: The frequency of reproducible research practices, defined as (1) effect estimate and measure of precision (eg, hazard ratio with 95% confidence interval); (2) clear list of studies included for each analysis; and (3) for subgroup and sensitivity analyses, it must be clear which studies were included in each group or level. RESULTS: We identified 1124 potential systematic reviews, and 154 meta-analyses comprising 3696 meta-analytic effect size estimates were included. Only 2375 of the 3696 meta-analytic estimates (64.3%), including subgroup and sensitivity analyses, were reproducible in theory. Forest plots appear to improve the reproducibility of meta-analyses. All meta-analytic estimates were reproducible in theory in 100 systematic reviews (64.9%), and in 15 systematic reviews (9.7%), no meta-analytic estimates could potentially be reproduced. Data were said to be imputed in 29 meta-analyses, but none specified which data. Only 1 meta-analysis included a link to an online data set. CONCLUSIONS AND RELEVANCE: More reproducible research practices are needed in oncology meta-analyses, as suggested by those that are cited by the NCCN. Reporting meta-analyses in forest plots and requirements for full data sharing are recommended.
重要性:可重复的研究实践对生物医学研究至关重要,因为这些实践能促进产生可靠的证据。在系统评价和荟萃分析中,可重复的研究实践可确保用于指导患者护理的汇总效应稳定且可靠。 目的:评估按癌症部位分类的49项美国国立综合癌症网络(NCCN)癌症治疗指南所引用的肿瘤学系统评价中荟萃分析在理论上的可重复性,并评估遵循系统评价和荟萃分析优先报告项目(PRISMA)指南的Cochrane评价或系统评价是否采用了更多可重复的研究实践。 设计、设置和参与者:对所有至少有1项荟萃分析且至少有1项纳入的随机临床试验(RCT)的系统评价进行横断面调查,这些系统评价被NCCN按癌症部位分类的治疗指南所引用。我们扫描了所有NCCN指南(n = 49)的参考文献列表,以查找潜在的系统评价和荟萃分析。对所有检索到的研究进行筛选,并独立且一式两份地提取数据。分析于2018年5月6日至2019年1月28日进行。 主要结局和指标:可重复研究实践的频率,定义为:(1)效应估计值和精确性测量指标(如95%置信区间的风险比);(2)每次分析纳入研究的清晰列表;(3)对于亚组分析和敏感性分析,必须明确每组或每个水平纳入了哪些研究。 结果:我们识别出1124项潜在的系统评价,纳入了154项荟萃分析,其中包含3,696个荟萃分析效应量估计值。在3,696个荟萃分析估计值(包括亚组分析和敏感性分析)中,理论上只有2,375个(64.3%)是可重复的。森林图似乎能提高荟萃分析的可重复性。在100项系统评价(64.9%)中,所有荟萃分析估计值在理论上都是可重复的,而在15项系统评价(9.7%)中,没有任何荟萃分析估计值可能被重复。在29项荟萃分析中提到数据是估算的,但均未指明是哪些数据。只有1项荟萃分析包含指向在线数据集的链接。 结论和相关性:如NCCN所引用的那些研究所示,肿瘤学荟萃分析需要更多可重复的研究实践。建议采用森林图报告荟萃分析并要求完全共享数据。
Cochrane Database Syst Rev. 2022-2-1
Early Hum Dev. 2020-11
Cochrane Database Syst Rev. 2014-10-1
J Clin Epidemiol. 2017-11-4
Scand J Work Environ Health. 2017-12-8
Cochrane Evid Synth Methods. 2023-4-10
Cochrane Evid Synth Methods. 2023-4-10
Expert Rev Precis Med Drug Dev. 2024
Nat Hum Behav. 2017-1-10
J Clin Epidemiol. 2017-11-4
Crit Rev Oncol Hematol. 2017-2-24
Elife. 2017-1-19
Syst Rev. 2016-12-5