Tang Gengyan, Cai Hao
Werklund School of Education, University of Calgary, Calgary, Alberta, Canada.
The First Affiliated Hospital, Chongqing Medical University, Chongqing, China.
JAMA Netw Open. 2025 Jun 2;8(6):e2515160. doi: 10.1001/jamanetworkopen.2025.15160.
Systematic reviews are the criterion standard for evidence synthesis in the life sciences, yet their reliability and integrity are threatened by citation contamination from fabricated publications produced by paper mills. Despite growing awareness, the extent and implications of this issue remain unclear.
To analyze the prevalence, characteristics, affected subject areas, and citation patterns of retracted paper mill articles cited in systematic reviews.
DESIGN, SETTING, AND PARTICIPANTS: This cross-sectional study analyzed systematic reviews published between 2013 and 2024, indexed in Web of Science (WoS). References were matched against the Retraction Watch dataset, and full texts were reviewed to identify retracted paper mill articles incorporated into the evidence synthesis.
The study assessed (1) contamination prevalence, defined as the proportion of systematic reviews incorporating retracted paper mill articles into the evidence synthesis; (2) geographic distribution of citing authors according to institutional affiliations; (3) citation timing and trends, including the time lag between incorporation and article retraction; (4) affected research areas, categorized by WoS subject classifications; and (5) citation patterns, including highly contaminated reviews (≥3 incorporations of retracted articles).
Of the total of 200 000 systematic reviews, 299 incorporated at least 1 retracted paper mill article into the evidence synthesis (contamination rate, 0.15%). Among them, 256 (85.6%) included a single retracted article, and 43 (14.4%) included multiple such articles. Of 1802 author affiliations associated with the contaminated reviews, 660 (36.6%) were from institutions in China. Of 385 total citations, 124 (32.2%) occurred after retraction, including 13 occurring more than 500 days after the retraction date. Oncology was the most affected field (48 of 299 [16.1%]). Five reviews each included 5 or more retracted articles, all published in journals under questionable publishers.
In this cross-sectional study of life sciences systematic reviews, contamination remained low but increased over time, posing a risk to research integrity. Continued citation of retracted articles, even after retraction, highlights the need for rigorous screening practices. Correcting contaminated reviews and developing automated detection tools are essential to preserving the credibility of systematic reviews.
系统评价是生命科学领域证据综合的标准规范，但它们的可靠性和完整性受到来自论文工厂炮制的虚假出版物的引文污染的威胁。尽管人们的认识不断提高，但这个问题的严重程度和影响仍不明确。
分析系统评价中被引用的撤回的论文工厂文章的流行情况、特征、受影响的学科领域和引文模式。
设计、设置和参与者：这项横断面研究分析了2013年至2024年期间发表并被科学引文索引（WoS）收录的系统评价。参考文献与撤回观察数据集进行匹配，并对全文进行审查，以识别纳入证据综合的撤回的论文工厂文章。
该研究评估了（1）污染流行率，定义为将撤回的论文工厂文章纳入证据综合的系统评价的比例；（2）根据机构隶属关系对引用作者的地理分布；（3）引文时间和趋势，包括纳入和文章撤回之间的时间间隔；（4）受影响的研究领域，按WoS学科分类进行分类；（5）引文模式，包括高污染评价（≥3次纳入撤回文章）。
在总共200,000篇系统评价中，299篇将至少1篇撤回的论文工厂文章纳入了证据综合（污染率为0.15%）。其中，256篇（占85.6%）包含1篇撤回文章，43篇（占14.4%）包含多篇此类文章。在与受污染评价相关的1802个作者隶属关系中，660个（占36.6%）来自中国的机构。在总共385次引文中，124次（占32.2%）发生在撤回之后，其中13次发生在撤回日期之后500多天。肿瘤学是受影响最严重的领域（299篇中有48篇[占16.1%]）。有5篇评价各自包含5篇或更多撤回文章，这些评价均发表在可疑出版商旗下的期刊上。
在这项关于生命科学系统评价的横断面研究中，污染程度仍然较低，但随时间有所增加，对研究诚信构成风险。即使在文章撤回后仍持续引用，凸显了严格筛选做法的必要性。纠正受污染评价并开发自动检测工具对于维护系统评价的可信度至关重要。
