Lin Frank P, Tran Minh, Thavaneswaran Subotheni, Grady John P, Kansara Maya, Chan Jeffrey, Ballinger Mandy L, Simes John, Thomas David M
Garvan Institute of Medical Research, Sydney, Australia.
NHMRC Clinical Trials Centre, University of Sydney, Australia.
Stud Health Technol Inform. 2025 Aug 7;329:248-252. doi: 10.3233/SHTI250839.
The curation of precision oncology knowledge bases requires intensive screening of scientific literature. Large Language Models (LLMs) offer potential to assist with knowledge extraction and curation through automated abstract screening, though their utility in this context remains unexplored. We developed a decision tree-based pipeline utilising LLMs to automate literature screening for precision oncology knowledge bases. The system employs structured prompts to evaluate biomarker-therapeutic relationships across 8,012 abstracts from 25 oncology journals published in 2024. Performance was assessed against expert review using standard evaluation metrics; the pipeline achieved an overall accuracy of 92.9% (sensitivity [recall] 87.1%, specificity 93.7%) in shortlisting 1,328 candidate abstracts, of which 888 are considered relevant by experts, delivering a 5.26-fold enrichment in classification precision. Analysis of correctly identified abstracts revealed comprehensive coverage of key molecular targets, drug classes (immunotherapies and targeted therapies), and relevant cancer types. This pipeline automates literature triage for precision oncology knowledge bases for relevance, maintaining high specificity and potentially reducing manual screening burden whilst offering a scalable framework for sustainable knowledge base maintenance.
精准肿瘤学知识库的整理需要对科学文献进行密集筛选。大语言模型(LLMs)有潜力通过自动摘要筛选协助知识提取和整理,不过其在这方面的效用仍未得到探索。我们开发了一种基于决策树的流程,利用大语言模型为精准肿瘤学知识库实现文献筛选自动化。该系统采用结构化提示来评估来自2024年出版的25种肿瘤学期刊的8012篇摘要中的生物标志物 - 治疗关系。使用标准评估指标对照专家评审来评估性能;该流程在筛选出1328篇候选摘要时总体准确率达到92.9%(敏感性[召回率]87.1%,特异性93.7%),其中专家认为888篇相关,分类精度提高了5.26倍。对正确识别的摘要进行分析发现,该流程全面涵盖了关键分子靶点、药物类别(免疫疗法和靶向疗法)以及相关癌症类型。此流程为精准肿瘤学知识库实现文献分类自动化以确定相关性,保持了高特异性,并有可能减轻人工筛选负担,同时为可持续的知识库维护提供了一个可扩展的框架。