Xue Yuting, Zhang Dongxu, Jia Liye, Yang Wanting, Zhao Juanjuan, Qiang Yan, Wang Long, Qiao Ying, Yue Huajie
College of Information and Computer, Taiyuan University of Technology, Taiyuan, Shanxi, China.
School of Software, Taiyuan University of Technology, Taiyuan, Shanxi, China.
PLoS One. 2024 Mar 11;19(3):e0297331. doi: 10.1371/journal.pone.0297331. eCollection 2024.
KRAS is a pathogenic gene frequently implicated in non-small cell lung cancer (NSCLC). However, biopsy as a diagnostic method has practical limitations. Therefore, it is important to accurately determine the mutation status of the KRAS gene non-invasively by combining NSCLC CT images and genetic data for early diagnosis and subsequent targeted therapy of patients. This paper proposes a Semi-supervised Multimodal Multiscale Attention Model (S2MMAM). S2MMAM comprises a Supervised Multilevel Fusion Segmentation Network (SMF-SN) and a Semi-supervised Multimodal Fusion Classification Network (S2MF-CN). S2MMAM facilitates the execution of the classification task by transferring the useful information captured in SMF-SN to the S2MF-CN to improve the model prediction accuracy. In SMF-SN, we propose a Triple Attention-guided Feature Aggregation module for obtaining segmentation features that incorporate high-level semantic abstract features and low-level semantic detail features. Segmentation features provide pre-guidance and key information expansion for S2MF-CN. S2MF-CN shares the encoder and decoder parameters of SMF-SN, which enables S2MF-CN to obtain rich classification features. S2MF-CN uses the proposed Intra and Inter Mutual Guidance Attention Fusion (I2MGAF) module to first guide segmentation and classification feature fusion to extract hidden multi-scale contextual information. I2MGAF then guides the multidimensional fusion of genetic data and CT image data to compensate for the lack of information in single modality data. S2MMAM achieved 83.27% AUC and 81.67% accuracy in predicting KRAS gene mutation status in NSCLC. This method uses medical image CT and genetic data to effectively improve the accuracy of predicting KRAS gene mutation status in NSCLC.
KRAS是一种经常与非小细胞肺癌(NSCLC)相关的致病基因。然而,活检作为一种诊断方法存在实际局限性。因此,通过结合NSCLC的CT图像和基因数据来无创地准确确定KRAS基因的突变状态,对于患者的早期诊断和后续靶向治疗非常重要。本文提出了一种半监督多模态多尺度注意力模型(S2MMAM)。S2MMAM由一个监督多级别融合分割网络(SMF-SN)和一个半监督多模态融合分类网络(S2MF-CN)组成。S2MMAM通过将SMF-SN中捕获的有用信息转移到S2MF-CN来促进分类任务的执行,以提高模型预测准确性。在SMF-SN中,我们提出了一个三重注意力引导特征聚合模块,用于获取包含高级语义抽象特征和低级语义细节特征的分割特征。分割特征为S2MF-CN提供预指导和关键信息扩展。S2MF-CN共享SMF-SN的编码器和解码器参数,这使得S2MF-CN能够获得丰富的分类特征。S2MF-CN使用所提出的内部和相互引导注意力融合(I2MGAF)模块,首先引导分割和分类特征融合以提取隐藏的多尺度上下文信息。I2MGAF然后引导基因数据和CT图像数据的多维度融合,以弥补单模态数据中信息的不足。S2MMAM在预测NSCLC中KRAS基因突变状态时,AUC达到83.27%,准确率达到81.67%。该方法利用医学图像CT和基因数据有效地提高了预测NSCLC中KRAS基因突变状态的准确性。