Zhang Yongtao, Li Haimei, Du Jie, Qin Jing, Wang Tianfu, Chen Yue, Liu Bing, Gao Wenwen, Ma Guolin, Lei Baiying
IEEE Trans Med Imaging. 2021 Jun;40(6):1618-1631. doi: 10.1109/TMI.2021.3062902. Epub 2021 Jun 1.
Automatic gastric tumor segmentation and lymph node (LN) classification not only can assist radiologists in reading images, but also provide image-guided clinical diagnosis and improve diagnosis accuracy. However, due to the inhomogeneous intensity distribution of gastric tumor and LN in CT scans, the ambiguous/missing boundaries, and highly variable shapes of gastric tumor, it is quite challenging to develop an automatic solution. To comprehensively address these challenges, we propose a novel 3D multi-attention guided multi-task learning network for simultaneous gastric tumor segmentation and LN classification, which makes full use of the complementary information extracted from different dimensions, scales, and tasks. Specifically, we tackle task correlation and heterogeneity with the convolutional neural network consisting of scale-aware attention-guided shared feature learning for refined and universal multi-scale features, and task-aware attention-guided feature learning for task-specific discriminative features. This shared feature learning is equipped with two types of scale-aware attention (visual attention and adaptive spatial attention) and two stage-wise deep supervision paths. The task-aware attention-guided feature learning comprises a segmentation-aware attention module and a classification-aware attention module. The proposed 3D multi-task learning network can balance all tasks by combining segmentation and classification loss functions with weight uncertainty. We evaluate our model on an in-house CT images dataset collected from three medical centers. Experimental results demonstrate that our method outperforms the state-of-the-art algorithms, and obtains promising performance for tumor segmentation and LN classification. Moreover, to explore the generalization for other segmentation tasks, we also extend the proposed network to liver tumor segmentation in CT images of the MICCAI 2017 Liver Tumor Segmentation Challenge. Our implementation is released at https://github.com/infinite-tao/MA-MTLN.
自动胃肿瘤分割和淋巴结（LN）分类不仅可以辅助放射科医生阅读图像，还能提供图像引导的临床诊断并提高诊断准确性。然而，由于CT扫描中胃肿瘤和淋巴结的强度分布不均匀、边界模糊/缺失以及胃肿瘤形状高度可变，开发一种自动解决方案颇具挑战性。为了全面应对这些挑战，我们提出了一种新颖的3D多注意力引导多任务学习网络，用于同时进行胃肿瘤分割和淋巴结分类，该网络充分利用了从不同维度、尺度和任务中提取的互补信息。具体而言，我们通过由尺度感知注意力引导的共享特征学习组成的卷积神经网络来处理任务相关性和异质性，以获得精细和通用的多尺度特征，以及通过任务感知注意力引导的特征学习来获得特定于任务的判别特征。这种共享特征学习配备了两种类型的尺度感知注意力（视觉注意力和自适应空间注意力）以及两条阶段式深度监督路径。任务感知注意力引导的特征学习包括一个分割感知注意力模块和一个分类感知注意力模块。所提出的3D多任务学习网络可以通过将分割和分类损失函数与权重不确定性相结合来平衡所有任务。我们在从三个医疗中心收集的内部CT图像数据集上评估我们的模型。实验结果表明，我们的方法优于现有算法，并且在肿瘤分割和淋巴结分类方面取得了有前景的性能。此外，为了探索对其他分割任务的泛化能力，我们还将所提出的网络扩展到了MICCAI 2017肝脏肿瘤分割挑战赛CT图像中的肝脏肿瘤分割。我们的实现可在https://github.com/infinite-tao/MA-MTLN上获取。
