Yang Junlin, Dvornek Nicha C, Zhang Fan, Chapiro Julius, Lin MingDe, Duncan James S
Department of Biomedical Engineering, Yale University, New Haven, CT, USA.
Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, USA.
Med Image Comput Comput Assist Interv. 2019 Oct;11765:255-263. doi: 10.1007/978-3-030-32245-8_29. Epub 2019 Oct 10.
A deep learning model trained on some labeled data from a certain source domain generally performs poorly on data from different target domains due to domain shifts. Unsupervised domain adaptation methods address this problem by alleviating the domain shift between the labeled source data and the unlabeled target data. In this work, we achieve cross-modality domain adaptation, i.e. between CT and MRI images, via disentangled representations. Compared to learning a one-to-one mapping as the state-of-art CycleGAN, our model recovers a manyto-many mapping between domains to capture the complex cross-domain relations. It preserves semantic feature-level information by finding a shared content space instead of a direct pixelwise style transfer. Domain adaptation is achieved in two steps. First, images from each domain are embedded into two spaces, a shared domain-invariant content space and a domain-specific style space. Next, the representation in the content space is extracted to perform a task. We validated our method on a cross-modality liver segmentation task, to train a liver segmentation model on CT images that also performs well on MRI. Our method achieved Dice Similarity Coefficient (DSC) of 0.81, outperforming a CycleGAN-based method of 0.72. Moreover, our model achieved good generalization to joint-domain learning, in which unpaired data from different modalities are jointly learned to improve the segmentation performance on each individual modality. Lastly, under a multi-modal target domain with significant diversity, our approach exhibited the potential for diverse image generation and remained effective with DSC of 0.74 on multi-phasic MRI while the CycleGAN-based method performed poorly with a DSC of only 0.52.
在来自某个源域的一些标注数据上训练的深度学习模型，由于域偏移，通常在来自不同目标域的数据上表现不佳。无监督域适应方法通过减轻标注源数据和未标注目标数据之间的域偏移来解决这个问题。在这项工作中，我们通过解缠表示实现了跨模态域适应，即CT图像和MRI图像之间的跨模态域适应。与学习一对一映射的当前最优CycleGAN相比，我们的模型恢复了域之间的多对多映射，以捕捉复杂的跨域关系。它通过找到一个共享的内容空间而不是直接的逐像素风格迁移来保留语义特征级信息。域适应分两步实现。首先，将来自每个域的图像嵌入到两个空间中，一个共享的域不变内容空间和一个特定于域的风格空间。接下来，提取内容空间中的表示以执行任务。我们在一个跨模态肝脏分割任务上验证了我们的方法，即在CT图像上训练一个肝脏分割模型，该模型在MRI图像上也表现良好。我们的方法实现了0.81的骰子相似系数（DSC），优于基于CycleGAN的方法（0.72）。此外，我们的模型在联合域学习方面实现了良好的泛化，其中来自不同模态的未配对数据被联合学习，以提高每个单独模态上的分割性能。最后，在具有显著多样性的多模态目标域下，我们的方法展现了多样化图像生成的潜力，并且在多期MRI上以0.74的DSC保持有效，而基于CycleGAN的方法表现不佳，DSC仅为0.52。
