Xu Chang, Fan Shiqi, Tian Yuan, Liu Fuchen, Furuya-Kanamori Luis, Clark Justin, Zhang Chao, Li Sheng, Lin Lifeng, Chu Haitao, Li Sheyu, Golder Su, Loke Yoon, Vohra Sunita, Glasziou Paul, Doi Suhail A, Liu Hui
Proof of Concept Center, Eastern Hepatobiliary Surgery Hospital, Third Affiliated Hospital, Second Military Medical University, Naval Medical University, Shanghai, China.
The Third Department of Hepatic Surgery, Eastern Hepatobiliary Surgery Hospital, Third Affiliated Hospital, Second Military Medical University, Naval Medical University, Shanghai, China
BMJ. 2025 Apr 23;389:e082068. doi: 10.1136/bmj-2024-082068.
To investigate the impact of retracted trials on the production and use of healthcare evidence in the evidence ecosystem.
Retrospective cohort study based on forward citation searching.
Retraction Watch up to 5 November 2024.
Randomised controlled trials in humans that were retracted for any reason.
Forward citation searching via Google Scholar and Scopus was used to identify evidence synthesis research (21 November 2024) that quantitatively incorporated retracted trials. Data were independently extracted by two groups of researchers. The results of meta-analyses were updated after exclusion of the retracted trials. The proportions of meta-analyses that changed direction of the pooled effect and/or the significance of the P value were estimated. A generalised linear mixed model was used to investigate the association between the number of included studies and the impact, measured by odds ratio and 95% confidence interval (CI). The impact of distorted evidence on clinical practice guidelines was also investigated on the basis of citation searching.
The searches identified 1330 retracted trials and 847 systematic reviews that quantitatively synthesised retracted trials, with a total of 3902 meta-analyses that could be replicated. After the potential clustering effects were accounted for, the exclusion of the retracted trials led to a change in the direction of the pooled effect in 8.4% (95% CI 6.8% to 10.1%), in its statistical significance in 16.0% (14.2% to 17.9%), and in both direction and significance in 3.9% (2.5% to 5.2%) and a >50% change in the magnitude of the effect in 15.7% (13.5% to 17.9%). An obvious non-linear association existed between the number of included studies and the impact on the results, with a lower number of studies having higher impact (eg, for 10 studies versus ≥20 studies, change of direction: odds ratio 2.63, 95% CI 1.29 to 5.38; P<0.001). Evidence from 68 systematic reviews with conclusions distorted by retracted trials was used in 157 guideline documents.
Retracted trials have a substantial impact on the evidence ecosystem, including evidence synthesis, clinical practice guidelines, and evidence based clinical practice. Evidence generators, synthesisers, and users must pay attention to this problem, and feasible approaches that assist with easier identification and correction of such potential contamination are needed.
Open Science Framework (https://osf.io/7eazq/).
探讨撤稿试验对证据生态系统中医疗保健证据的产生和使用的影响。
基于向前引文检索的回顾性队列研究。
截至2024年11月5日的撤稿观察网。
因任何原因撤稿的人体随机对照试验。
通过谷歌学术和Scopus进行向前引文检索，以识别在2024年11月21日定量纳入撤稿试验的证据综合研究。数据由两组研究人员独立提取。在排除撤稿试验后更新荟萃分析结果。估计改变合并效应方向和/或P值显著性的荟萃分析比例。使用广义线性混合模型研究纳入研究数量与影响之间的关联，以比值比和95%置信区间（CI）衡量。还基于引文检索研究了扭曲证据对临床实践指南的影响。
检索确定了1330项撤稿试验和847项定量综合撤稿试验的系统评价，共有3902项荟萃分析可重复。在考虑潜在的聚类效应后，排除撤稿试验导致合并效应方向改变8.4%（95%CI 6.8%至10.1%），其统计学显著性改变16.0%（14.2%至17.9%），方向和显著性均改变3.9%（2.5%至5.2%），效应大小改变>50%的占15.7%（13.5%至17.9%）。纳入研究数量与对结果的影响之间存在明显的非线性关联，研究数量较少时影响较大（例如，对于10项研究与≥20项研究，方向改变：比值比2.63，95%CI 1.29至5.38；P<0.001）。157份指南文件中使用了68项系统评价的证据，这些系统评价的结论因撤稿试验而扭曲。
撤稿试验对证据生态系统有重大影响，包括证据综合、临床实践指南和循证临床实践。证据产生者、综合者和使用者必须关注这个问题，并且需要可行的方法来帮助更轻松地识别和纠正这种潜在的污染。
开放科学框架（https://osf.io/7eazq/）。
