Liang Junjie, Yang Cihui, Zeng Mengjie, Wang Xixi
School of Information Engineering, Nanchang Hangkong University, Nanchang, China.
Quant Imaging Med Surg. 2022 Apr;12(4):2397-2415. doi: 10.21037/qims-21-919.
Medical image segmentation plays a vital role in computer-aided diagnosis (CAD) systems. Both convolutional neural networks (CNNs) with strong local information extraction capacities and transformers with excellent global representation capacities have achieved remarkable performance in medical image segmentation. However, because of the semantic differences between local and global features, how to combine convolution and transformers effectively is an important challenge in medical image segmentation.
In this paper, we proposed TransConver, a U-shaped segmentation network based on convolution and transformer for automatic and accurate brain tumor segmentation in MRI images. Unlike the recently proposed transformer and convolution based models, we proposed a parallel module named transformer-convolution inception (TC-inception), which extracts local and global information via convolution blocks and transformer blocks, respectively, and integrates them by a cross-attention fusion with global and local feature (CAFGL) mechanism. Meanwhile, the improved skip connection structure named skip connection with cross-attention fusion (SCCAF) mechanism can alleviate the semantic differences between encoder features and decoder features for better feature fusion. In addition, we designed 2D-TransConver and 3D-TransConver for 2D and 3D brain tumor segmentation tasks, respectively, and verified the performance and advantage of our model through brain tumor datasets.
We trained our model on 335 cases from the training dataset of MICCAI BraTS2019 and evaluated the model's performance based on 66 cases from MICCAI BraTS2018 and 125 cases from MICCAI BraTS2019. Our TransConver achieved the best average Dice score of 83.72% and 86.32% on BraTS2019 and BraTS2018, respectively.
We proposed a transformer and convolution parallel network named TransConver for brain tumor segmentation. The TC-Inception module effectively extracts global information while retaining local details. The experimental results demonstrated that good segmentation requires the model to extract local fine-grained details and global semantic information simultaneously, and our TransConver effectively improves the accuracy of brain tumor segmentation.
医学图像分割在计算机辅助诊断（CAD）系统中起着至关重要的作用。具有强大局部信息提取能力的卷积神经网络（CNN）和具有出色全局表示能力的变换器在医学图像分割中都取得了显著的性能。然而，由于局部和全局特征之间的语义差异，如何有效地结合卷积和变换器是医学图像分割中的一个重要挑战。
在本文中，我们提出了TransConver，一种基于卷积和变换器的U形分割网络，用于在MRI图像中自动准确地分割脑肿瘤。与最近提出的基于变换器和卷积的模型不同，我们提出了一个名为变换器-卷积 inception（TC-inception）的并行模块，它分别通过卷积块和变换器块提取局部和全局信息，并通过具有全局和局部特征的交叉注意力融合（CAFGL）机制将它们集成。同时，名为具有交叉注意力融合的跳跃连接（SCCAF）机制的改进跳跃连接结构可以缓解编码器特征和解码器特征之间的语义差异，以实现更好的特征融合。此外，我们分别为2D和3D脑肿瘤分割任务设计了2D-TransConver和3D-TransConver，并通过脑肿瘤数据集验证了我们模型的性能和优势。
我们在MICCAI BraTS2019训练数据集的335个病例上训练了我们的模型，并基于MICCAI BraTS2018的66个病例和MICCAI BraTS2019的125个病例评估了模型的性能。我们的TransConver在BraTS2019和BraTS2018上分别取得了83.72%和86.32%的最佳平均骰子分数。
我们提出了一种名为TransConver的变换器和卷积并行网络用于脑肿瘤分割。TC-Inception模块在保留局部细节的同时有效地提取全局信息。实验结果表明，良好的分割需要模型同时提取局部细粒度细节和全局语义信息，并且我们的TransConver有效地提高了脑肿瘤分割的准确性。
