Sun Huajun, Wei Jia, Yuan Wenguang, Li Rui
South China University of Technology, Guangzhou, 510006, China.
Huawei Cloud BU EI Innovation Laboratory, Dongguan, 523000, China.
Comput Biol Med. 2024 Jun;176:108570. doi: 10.1016/j.compbiomed.2024.108570. Epub 2024 May 8.
The two major challenges to deep-learning-based medical image segmentation are multi-modality and a lack of expert annotations. Existing semi-supervised segmentation models can mitigate the problem of insufficient annotations by utilizing a small amount of labeled data. However, most of these models are limited to single-modal data and cannot exploit the complementary information from multi-modal medical images. A few semi-supervised multi-modal models have been proposed recently, but they have rigid structures and require additional training steps for each modality. In this work, we propose a novel flexible method, semi-supervised multi-modal medical image segmentation with unified translation (SMSUT), and a unique semi-supervised procedure that can leverage multi-modal information to improve the semi-supervised segmentation performance. Our architecture capitalizes on unified translation to extract complementary information from multi-modal data which compels the network to focus on the disparities and salient features among each modality. Furthermore, we impose constraints on the model at both pixel and feature levels, to cope with the lack of annotation information and the diverse representations within semi-supervised multi-modal data. We introduce a novel training procedure tailored for semi-supervised multi-modal medical image analysis, by integrating the concept of conditional translation. Our method has a remarkable ability for seamless adaptation to varying numbers of distinct modalities in the training data. Experiments show that our model exceeds the semi-supervised segmentation counterparts in the public datasets which proves our network's high-performance capabilities and the transferability of our proposed method. The code of our method will be openly available at https://github.com/Sue1347/SMSUT-MedicalImgSegmentation.
基于深度学习的医学图像分割面临的两大挑战是多模态性和缺乏专家标注。现有的半监督分割模型可以通过利用少量标注数据来缓解标注不足的问题。然而，这些模型大多局限于单模态数据，无法利用多模态医学图像的互补信息。最近已经提出了一些半监督多模态模型，但它们结构僵化，每种模态都需要额外的训练步骤。在这项工作中，我们提出了一种新颖的灵活方法，即具有统一转换的半监督多模态医学图像分割（SMSUT），以及一种独特的半监督程序，该程序可以利用多模态信息来提高半监督分割性能。我们的架构利用统一转换从多模态数据中提取互补信息，这迫使网络关注每种模态之间的差异和显著特征。此外，我们在像素和特征层面都对模型施加约束，以应对半监督多模态数据中缺乏标注信息和多样表示的问题。我们通过整合条件转换的概念，引入了一种专门为半监督多模态医学图像分析量身定制的新颖训练程序。我们的方法具有出色的能力，能够无缝适应训练数据中不同模态数量的变化。实验表明，我们的模型在公共数据集中超过了半监督分割的同类模型，证明了我们网络的高性能能力以及我们所提方法的可迁移性。我们方法的代码将在https://github.com/Sue1347/SMSUT-MedicalImgSegmentation上公开提供。
