Hooshangnejad Hamed, Huang Gaofeng, Kelly Katelyn, Feng Xue, Luo Yi, Zhang Rui, Xu Ziyue, Chen Quan, Ding Kai
Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD 21218, USA.
Department of Radiation Oncology and Molecular Radiation Sciences, Johns Hopkins University, Baltimore, MD 21287, USA.
Cancers (Basel). 2024 Dec 6;16(23):4097. doi: 10.3390/cancers16234097.
BACKGROUND/OBJECTIVES: Lung cancer is a devastating disease with the highest mortality rate among cancer types. Over 60% of non-small cell lung cancer (NSCLC) patients, accounting for 87% of lung cancer diagnoses, require radiation therapy. Rapid treatment initiation significantly increases the patient's survival rate and reduces the mortality rate. Accurate tumor segmentation is a critical step in diagnosing and treating NSCLC. Manual segmentation is time- and labor-consuming and causes delays in treatment initiation. Although many lung nodule detection methods, including deep learning-based models, have been proposed. Most of these methods still have a long-standing problem of high false positives (FPs).
Here, we developed an electronic health record (EHR)-guided lung tumor auto-segmentation called EXACT-Net (EHR-enhanced eXACtitude in Tumor segmentation), where the extracted information from EHRs using a pre-trained large language model (LLM) was used to remove the FPs and keep the TP nodules only.
The auto-segmentation model was trained on NSCLC patients' computed tomography (CT), and the pre-trained LLM was used with the zero-shot learning approach. Our approach resulted in a 250% boost in successful nodule detection using the data from ten NSCLC patients treated in our institution.
We demonstrated that combining vision-language information in EXACT-Net multi-modal AI framework greatly enhances the performance of vision only models, paving the road to multimodal AI framework for medical image processing.
背景/目的：肺癌是一种具有毁灭性的疾病，在各类癌症中死亡率最高。超过60%的非小细胞肺癌（NSCLC）患者（占肺癌确诊病例的87%）需要接受放射治疗。快速开始治疗可显著提高患者生存率并降低死亡率。准确的肿瘤分割是NSCLC诊断和治疗的关键步骤。手动分割既耗时又费力，会导致治疗开始延迟。尽管已经提出了许多肺结节检测方法，包括基于深度学习的模型。但这些方法大多仍存在长期存在的高假阳性（FP）问题。
在此，我们开发了一种名为EXACT-Net（肿瘤分割中EHR增强的精确性）的电子健康记录（EHR）引导的肺肿瘤自动分割方法，其中使用预训练的大语言模型（LLM）从EHR中提取的信息用于去除FP，仅保留真阳性（TP）结节。
自动分割模型在NSCLC患者的计算机断层扫描（CT）上进行训练，预训练的LLM采用零样本学习方法。我们的方法使用我们机构治疗的10例NSCLC患者的数据，成功结节检测率提高了250%。
我们证明，在EXACT-Net多模态人工智能框架中结合视觉语言信息可大大提高仅视觉模型的性能，为医学图像处理的多模态人工智能框架铺平了道路。
