Amirrajab Sina, Al Khalil Yasmina, Lorenz Cristian, Weese Jürgen, Pluim Josien, Breeuwer Marcel
Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands.
Philips Research Laboratories, Hamburg, Germany.
Comput Med Imaging Graph. 2022 Oct;101:102123. doi: 10.1016/j.compmedimag.2022.102123. Epub 2022 Sep 11.
Synthesis of a large set of high-quality medical images with variability in anatomical representation and image appearance has the potential to provide solutions for tackling the scarcity of properly annotated data in medical image analysis research. In this paper, we propose a novel framework consisting of image segmentation and synthesis based on mask-conditional GANs for generating high-fidelity and diverse Cardiac Magnetic Resonance (CMR) images. The framework consists of two modules: i) a segmentation module trained using a physics-based simulated database of CMR images to provide multi-tissue labels on real CMR images, and ii) a synthesis module trained using pairs of real CMR images and corresponding multi-tissue labels, to translate input segmentation masks to realistic-looking cardiac images. The anatomy of synthesized images is based on labels, whereas the appearance is learned from the training images. We investigate the effects of the number of tissue labels, quantity of training data, and multi-vendor data on the quality of the synthesized images. Furthermore, we evaluate the effectiveness and usability of the synthetic data for a downstream task of training a deep-learning model for cardiac cavity segmentation in the scenarios of data replacement and augmentation. The results of the replacement study indicate that segmentation models trained with only synthetic data can achieve comparable performance to the baseline model trained with real data, indicating that the synthetic data captures the essential characteristics of its real counterpart. Furthermore, we demonstrate that augmenting real with synthetic data during training can significantly improve both the Dice score (maximum increase of 4%) and Hausdorff Distance (maximum reduction of 40%) for cavity segmentation, suggesting a good potential to aid in tackling medical data scarcity.
合成大量具有解剖结构表现和图像外观可变性的高质量医学图像，有可能为解决医学图像分析研究中注释数据匮乏的问题提供解决方案。在本文中，我们提出了一种新颖的框架，该框架由基于掩码条件生成对抗网络（GAN）的图像分割与合成组成，用于生成高保真且多样的心脏磁共振（CMR）图像。该框架由两个模块组成：i）一个分割模块，使用基于物理的CMR图像模拟数据库进行训练，以在真实CMR图像上提供多组织标签；ii）一个合成模块，使用真实CMR图像对及其对应的多组织标签进行训练，将输入的分割掩码转换为逼真的心脏图像。合成图像的解剖结构基于标签，而外观则从训练图像中学习。我们研究了组织标签数量、训练数据量和多供应商数据对合成图像质量的影响。此外，我们在数据替换和增强的场景下，评估了合成数据在训练用于心脏腔室分割的深度学习模型这一下游任务中的有效性和可用性。替换研究结果表明，仅使用合成数据训练的分割模型能够达到与使用真实数据训练的基线模型相当的性能，这表明合成数据捕捉到了其真实对应物的基本特征。此外，我们证明在训练期间用合成数据增强真实数据，可以显著提高腔室分割的骰子系数（最大提高4%）和豪斯多夫距离（最大降低40%），这表明在帮助解决医学数据匮乏问题方面具有良好的潜力。
