Gandomi Amir, Hasan Eusha, Chusid Jesse, Paul Subroto, Inra Matthew, Makhnevich Alex, Raoof Suhail, Silvestri Gerard, Bade Brett C, Cohen Stuart L
Northwell, New Hyde Park, NY, USA; Institute of Health System Science, Feinstein Institutes for Medical Research, Manhasset, NY, USA; Frank G. Zarb School of Business, Hofstra University, Hempstead, NY, USA.
Northwell, New Hyde Park, NY, USA; Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA.
Int J Med Inform. 2024 Nov;191:105580. doi: 10.1016/j.ijmedinf.2024.105580. Epub 2024 Jul 31.
Radiology scoring systems are critical to the success of lung cancer screening (LCS) programs, impacting patient care, adherence to follow-up, data management and reporting, and program evaluation. LungCT ScreeningReporting and Data System (Lung-RADS) is a structured radiology scoring system that provides recommendations for LCS follow-up that are utilized (a) in clinical care and (b) by LCS programs monitoring rates of adherence to follow-up. Thus, accurate reporting and reliable collection of Lung-RADS scores are fundamental components of LCS program evaluation and improvement. Unfortunately, due to variability in radiology reports, extraction of Lung-RADS scores is non-trivial, and best practices do not exist. The purpose of this project is to compare mechanisms to extract Lung-RADS scores from free-text radiology reports.
We retrospectively analyzed reports of LCS low-dose computed tomography (LDCT) examinations performed at a multihospital integrated healthcare network in New York State between January 2016 and July 2023. We compared three methods of Lung-RADS score extraction: manual physician entry at time of report creation, manual LCS specialist entry after report creation, and an internally developed, rule-based natural language processing (NLP) algorithm. Accuracy, recall, precision, and completeness (i.e., the proportion of LCS exams to which a Lung-RADS score has been assigned) were compared between the three methods.
The dataset includes 24,060 LCS examinations on 14,243 unique patients. The mean patient age was 65 years, and most patients were male (54 %) and white (75 %). Completeness rate was 65 %, 68 %, and 99 % for radiologists' manual entry, LCS specialists' entry, and NLP algorithm, respectively. Accuracy, recall, and precision were high across all extraction methods (>94 %), though the NLP-based approach was consistently higher than both manual entries in all metrics.
An NLP-based method of LCS score determination is an efficient and more accurate means of extracting Lung-RADS scores than manual review and data entry. NLP-based methods should be considered best practice for extracting structured Lung-RADS scores from free-text radiology reports.
放射学评分系统对于肺癌筛查（LCS）项目的成功至关重要，影响着患者护理、随访依从性、数据管理与报告以及项目评估。肺部CT筛查报告与数据系统（Lung-RADS）是一种结构化放射学评分系统，它为LCS随访提供建议，这些建议被用于（a）临床护理以及（b）由LCS项目监测随访依从率。因此，准确报告和可靠收集Lung-RADS分数是LCS项目评估与改进的基本组成部分。不幸的是，由于放射学报告的变异性，提取Lung-RADS分数并非易事，且不存在最佳实践方法。本项目的目的是比较从自由文本放射学报告中提取Lung-RADS分数的机制。
我们回顾性分析了2016年1月至2023年7月在纽约州一个多医院综合医疗网络中进行的LCS低剂量计算机断层扫描（LDCT）检查报告。我们比较了三种提取Lung-RADS分数的方法：报告创建时医生手动录入、报告创建后LCS专家手动录入以及一种内部开发的基于规则的自然语言处理（NLP）算法。比较了三种方法之间的准确性、召回率、精确率和完整性（即已分配Lung-RADS分数的LCS检查的比例）。
数据集包括对14243名独特患者进行的24060次LCS检查。患者平均年龄为65岁，大多数患者为男性（54%）且为白人（75%）。放射科医生手动录入、LCS专家录入和NLP算法的完整率分别为65%、68%和99%。所有提取方法的准确性、召回率和精确率都很高（>94%），不过基于NLP的方法在所有指标上始终高于两种手动录入方法。
基于NLP的LCS分数确定方法是一种比人工审核和数据录入更有效、更准确的提取Lung-RADS分数的方法。基于NLP的方法应被视为从自由文本放射学报告中提取结构化Lung-RADS分数的最佳实践方法。
