Fletcher Aaron H A, Stevenson Mark
School of Computer Science, The University of Sheffield, Regent Court, Sheffield, S1 4DP, UK.
Res Integr Peer Rev. 2025 Jun 11;10(1):9. doi: 10.1186/s41073-025-00168-w.
Retractions undermine the scientific record's reliability and can lead to the continued propagation of flawed research. This study aimed to (1) create a dataset aggregating retraction information with bibliographic metadata, (2) train and evaluate various machine learning approaches to predict article retractions, and (3) assess each feature's contribution to feature-based classifier performance using ablation studies.
An open-access dataset was developed by combining information from the Retraction Watch database and the OpenAlex API. Using a case-controlled design, retracted research articles were paired with non-retracted articles published in the same period. Traditional feature-based classifiers and models leveraging contextual language representations were then trained and evaluated. Model performance was assessed using accuracy, precision, recall, and the F1-score.
The Llama 3.2 base model achieved the highest overall accuracy. The Random Forest classifier achieved a precision of 0.687 for identifying non-retracted articles, while the Llama 3.2 base model reached a precision of 0.683 for identifying retracted articles. Traditional feature-based classifiers generally outperformed most contextual language models, except for the Llama 3.2 base model, which showed competitive performance across several metrics.
Although no single model excelled across all metrics, our findings indicate that machine learning techniques can effectively support the identification of retracted research. These results provide a foundation for developing automated tools to assist publishers and reviewers in detecting potentially problematic publications. Further research should focus on refining these models and investigating additional features to improve predictive performance.
Not applicable.
撤稿会破坏科学记录的可靠性，并可能导致有缺陷的研究持续传播。本研究旨在：（1）创建一个将撤稿信息与文献元数据聚合在一起的数据集；（2）训练和评估各种机器学习方法以预测文章撤稿情况；（3）使用消融研究评估每个特征对基于特征的分类器性能的贡献。
通过合并来自Retraction Watch数据库和OpenAlex API的信息，开发了一个开放获取的数据集。采用病例对照设计，将撤稿的研究文章与同期发表的未撤稿文章进行配对。然后训练和评估传统的基于特征的分类器以及利用上下文语言表示的模型。使用准确率、精确率、召回率和F1分数评估模型性能。
Llama 3.2基础模型总体准确率最高。随机森林分类器在识别未撤稿文章方面的精确率为0.687，而Llama 3.2基础模型在识别撤稿文章方面的精确率达到0.683。除Llama 3.2基础模型在几个指标上表现出有竞争力的性能外，传统的基于特征的分类器通常优于大多数上下文语言模型。
尽管没有一个模型在所有指标上都表现出色，但我们的研究结果表明，机器学习技术可以有效地支持对撤稿研究的识别。这些结果为开发自动化工具提供了基础，以协助出版商和审稿人检测潜在有问题的出版物。进一步的研究应侧重于改进这些模型并研究其他特征以提高预测性能。
不适用。
