Moore Christopher L, Socrates Vimig, Hesami Mina, Denkewicz Ryan P, Cavallo Joe J, Venkatesh Arjun K, Taylor R Andrew
Department of Emergency Medicine, Yale University, New Haven, Connecticut, USA.
Department of Biomedical Informatics and Data Science, Yale University, New Haven, Connecticut, USA.
Acad Emerg Med. 2025 Mar;32(3):274-283. doi: 10.1111/acem.15080. Epub 2025 Jan 17.
For emergency department (ED) patients, lung cancer may be detected early through incidental lung nodules (ILNs) discovered on chest CTs. However, there are significant errors in the communication and follow-up of incidental findings on ED imaging, particularly due to unstructured radiology reports. Natural language processing (NLP) can aid in identifying ILNs requiring follow-up, potentially reducing errors from missed follow-up. We sought to develop an open-access, three-step NLP pipeline specifically for this purpose.
This retrospective used a cohort of 26,545 chest CTs performed in three EDs from 2014 to 2021. Randomly selected chest CT reports were annotated by MD raters using Prodigy software to develop a stepwise NLP "pipeline" that first excluded prior or known malignancy, determined the presence of a lung nodule, and then categorized any recommended follow-up. NLP was developed using a RoBERTa large language model on the SpaCy platform and deployed as open-access software using Docker. After NLP development it was applied to 1000 CT reports that were manually reviewed to determine accuracy using accepted NLP metrics of precision (positive predictive value), recall (sensitivity), and F1 score (which balances precision and recall).
Precision, recall, and F1 score were 0.85, 0.71, and 0.77, respectively, for malignancy; 0.87, 0.83, and 0.85 for nodule; and 0.82, 0.90, and 0.85 for follow-up. Overall accuracy for follow-up in the absence of malignancy with a nodule present was 93.3%. The overall recommended follow-up rate was 12.4%, with 10.1% of patients having evidence of known or prior malignancy.
We developed an accurate, open-access pipeline to identify ILNs with recommended follow-up on ED chest CTs. While the prevalence of recommended follow-up is lower than some prior studies, it more accurately reflects the prevalence of truly incidental findings without prior or known malignancy. Incorporating this tool could reduce errors by improving the identification, communication, and tracking of ILNs.
对于急诊科（ED）患者，肺癌可能通过胸部CT发现的偶然肺结节（ILN）得以早期检测。然而，急诊影像检查中偶然发现的结果在沟通和后续跟进方面存在重大错误，尤其是由于放射学报告缺乏结构化。自然语言处理（NLP）有助于识别需要后续跟进的ILN，有可能减少因漏诊后续跟进而导致的错误。我们试图专门为此开发一个开放获取的三步NLP流程。
这项回顾性研究使用了2014年至2021年在三个急诊科进行的26545例胸部CT队列。随机选择的胸部CT报告由医学博士评估人员使用Prodigy软件进行注释，以开发一个逐步的NLP“流程”，该流程首先排除既往或已知的恶性肿瘤，确定肺结节的存在，然后对任何推荐的后续跟进进行分类。NLP是在SpaCy平台上使用RoBERTa大语言模型开发的，并使用Docker作为开放获取软件进行部署。在NLP开发完成后，将其应用于1000份CT报告，并通过使用公认的NLP精度（阳性预测值）、召回率（敏感性）和F1分数（平衡精度和召回率）指标进行人工审核以确定准确性。
恶性肿瘤的精度、召回率和F1分数分别为0.85、0.71和0.77；结节的精度、召回率和F1分数分别为0.87、0.83和0.85；后续跟进的精度、召回率和F1分数分别为0.82、0.90和0.85。在存在结节但无恶性肿瘤情况下的后续跟进总体准确率为93.3%。总体推荐后续跟进率为12.4%，10.1%的患者有已知或既往恶性肿瘤的证据。
我们开发了一个准确的、开放获取的流程，以识别急诊胸部CT上推荐进行后续跟进的ILN。虽然推荐后续跟进的患病率低于一些先前的研究，但它更准确地反映了无既往或已知恶性肿瘤的真正偶然发现的患病率。纳入这个工具可以通过改善ILN的识别、沟通和跟踪来减少错误。
