School of Computer Science and Engineering, North Minzu University, Yinchuan, 750021, China; Key Laboratory of Image and Graphics Intelligent Processing of State Ethnic Affairs Commission, North Minzu University, Yinchuan, 750021, China.
School of Computer Science and Engineering, North Minzu University, Yinchuan, 750021, China.
Comput Biol Med. 2023 Nov;166:107514. doi: 10.1016/j.compbiomed.2023.107514. Epub 2023 Sep 28.
Lung tumor PET and CT image fusion is a key technology in clinical diagnosis. However, the existing fusion methods are difficult to obtain fused images with high contrast, prominent morphological features, and accurate spatial localization. In this paper, an isomorphic Unet fusion model (GMRE-iUnet) for lung tumor PET and CT images is proposed to address the above problems. The main idea of this network is as following: Firstly, this paper constructs an isomorphic Unet fusion network, which contains two independent multiscale dual encoders Unet, it can capture the features of the lesion region, spatial localization, and enrich the morphological information. Secondly, a Hybrid CNN-Transformer feature extraction module (HCTrans) is constructed to effectively integrate local lesion features and global contextual information. In addition, the residual axial attention feature compensation module (RAAFC) is embedded into the Unet to capture fine-grained information as compensation features, which makes the model focus on local connections in neighboring pixels. Thirdly, a hybrid attentional feature fusion module (HAFF) is designed for multiscale feature information fusion, it aggregates edge information and detail representations using local entropy and Gaussian filtering. Finally, the experiment results on the multimodal lung tumor medical image dataset show that the model in this paper can achieve excellent fusion performance compared with other eight fusion models. In CT mediastinal window images and PET images comparison experiment, AG, EI, Q, SF, SD, and IE indexes are improved by 16.19%, 26%, 3.81%, 1.65%, 3.91% and 8.01%, respectively. GMRE-iUnet can highlight the information and morphological features of the lesion areas and provide practical help for the aided diagnosis of lung tumors.
肺部肿瘤 PET 和 CT 图像融合是临床诊断中的一项关键技术。然而，现有的融合方法很难获得对比度高、形态特征突出、空间定位准确的融合图像。本文提出了一种用于肺部肿瘤 PET 和 CT 图像的同构 U-Net 融合模型（GMRE-iUnet），以解决上述问题。该网络的主要思想如下：首先，构建了一个同构 U-Net 融合网络，该网络包含两个独立的多尺度双编码器 U-Net，可以捕获病变区域、空间定位和丰富形态信息的特征。其次，构建了一个混合 CNN-Transformer 特征提取模块（HCTrans），可以有效地整合局部病变特征和全局上下文信息。此外，将残差轴向注意力特征补偿模块（RAAFC）嵌入 U-Net 中，以捕获作为补偿特征的细粒度信息，使模型专注于相邻像素的局部连接。第三，设计了一个混合注意力特征融合模块（HAFF），用于多尺度特征信息融合，它使用局部熵和高斯滤波聚合边缘信息和细节表示。最后，在多模态肺部肿瘤医学图像数据集上的实验结果表明，与其他八个融合模型相比，本文提出的模型可以实现优异的融合性能。在 CT 纵隔窗图像和 PET 图像对比实验中，AG、EI、Q、SF、SD 和 IE 指数分别提高了 16.19%、26%、3.81%、1.65%、3.91%和 8.01%。GMRE-iUnet 可以突出病变区域的信息和形态特征，为肺部肿瘤的辅助诊断提供实际帮助。
