Department of Biomedical Engineering and Imaging Sciences, King's College London, Strand, London, WC2R 2LS, United Kingdom.
Department of Biomedical Engineering and Imaging Sciences, King's College London, Strand, London, WC2R 2LS, United Kingdom.
Med Image Anal. 2024 Oct;97:103278. doi: 10.1016/j.media.2024.103278. Epub 2024 Jul 18.
The last few years have seen a boom in using generative models to augment real datasets, as synthetic data can effectively model real data distributions and provide privacy-preserving, shareable datasets that can be used to train deep learning models. However, most of these methods are 2D and provide synthetic datasets that come, at most, with categorical annotations. The generation of paired images and segmentation samples that can be used in downstream, supervised segmentation tasks remains fairly uncharted territory. This work proposes a two-stage generative model capable of producing 2D and 3D semantic label maps and corresponding multi-modal images. We use a latent diffusion model for label synthesis and a VAE-GAN for semantic image synthesis. Synthetic datasets provided by this model are shown to work in a wide variety of segmentation tasks, supporting small, real datasets or fully replacing them while maintaining good performance. We also demonstrate its ability to improve downstream performance on out-of-distribution data.
过去几年中，使用生成模型来扩充真实数据集的方法越来越多，因为合成数据可以有效地模拟真实数据分布，并提供隐私保护、可共享的数据集，可用于训练深度学习模型。然而，这些方法大多是二维的，并且只能提供具有类别标注的合成数据集。在下游的监督分割任务中，生成可用于配对图像和分割样本的能力仍然是一个相当未知的领域。本文提出了一种两阶段的生成模型，能够生成 2D 和 3D 语义标签图以及相应的多模态图像。我们使用潜在扩散模型进行标签合成，使用 VAE-GAN 进行语义图像合成。该模型提供的合成数据集在各种分割任务中表现良好，支持小型真实数据集或完全替代它们，同时保持良好的性能。我们还证明了它能够提高对离群数据的下游性能。
