Hasso Plattner Institute, Prof.Dr. Helmert Street 2-3, Potsdam, Germany.
Massachusetts General Hospital and Harvard Medical School, 25 New Chardon St., Boston, MS, USA.
Int J Comput Assist Radiol Surg. 2020 Nov;15(11):1847-1858. doi: 10.1007/s11548-020-02254-4. Epub 2020 Sep 8.
Purpose The identification of abnormalities that are relatively rare within otherwise normal anatomy is a major challenge for deep learning in the semantic segmentation of medical images. The small number of samples of the minority classes in the training data makes the learning of optimal classification challenging, while the more frequently occurring samples of the majority class hamper the generalization of the classification boundary between infrequently occurring target objects and classes. In this paper, we developed a novel generative multi-adversarial network, called Ensemble-GAN, for mitigating this class imbalance problem in the semantic segmentation of abdominal images.Method The Ensemble-GAN framework is composed of a single-generator and a multi-discriminator variant for handling the class imbalance problem to provide a better generalization than existing approaches. The ensemble model aggregates the estimates of multiple models by training from different initializations and losses from various subsets of the training data. The single generator network analyzes the input image as a condition to predict a corresponding semantic segmentation image by use of feedback from the ensemble of discriminator networks. To evaluate the framework, we trained our framework on two public datasets, with different imbalance ratios and imaging modalities: the Chaos 2019 and the LiTS 2017.Result In terms of the F1 score, the accuracies of the semantic segmentation of healthy spleen, liver, and left and right kidneys were 0.93, 0.96, 0.90 and 0.94, respectively. The overall F1 scores for simultaneous segmentation of the lesions and liver were 0.83 and 0.94, respectively.Conclusion The proposed Ensemble-GAN framework demonstrated outstanding performance in the semantic segmentation of medical images in comparison with other approaches on popular abdominal imaging benchmarks. The Ensemble-GAN has the potential to segment abdominal images more accurately than human experts.
目的
在正常解剖结构中识别相对罕见的异常是医学图像语义分割中深度学习的主要挑战。训练数据中少数类别的样本数量较少，使得最优分类学习具有挑战性，而大多数类别中更频繁出现的样本则阻碍了在不常见目标对象和类别之间进行分类边界的泛化。在本文中，我们开发了一种新颖的生成式多对抗网络，称为 Ensemble-GAN，用于减轻腹部图像语义分割中的类不平衡问题。
方法
Ensemble-GAN 框架由单个生成器和多判别器变体组成，用于处理类不平衡问题，以提供比现有方法更好的泛化能力。集成模型通过从不同初始化和不同训练数据子集的损失进行训练，聚合多个模型的估计值。单个生成器网络将输入图像作为条件进行分析，通过来自判别器网络集合的反馈预测相应的语义分割图像。为了评估框架，我们在两个具有不同不平衡比和成像方式的公共数据集上训练我们的框架：Chaos 2019 和 LiTS 2017。
结果
就 F1 分数而言，健康脾、肝、左肾和右肾的语义分割准确率分别为 0.93、0.96、0.90 和 0.94。同时对病变和肝脏进行分割的整体 F1 分数分别为 0.83 和 0.94。
结论
与流行的腹部成像基准上的其他方法相比，所提出的 Ensemble-GAN 框架在医学图像的语义分割中表现出色。Ensemble-GAN 有可能比人类专家更准确地分割腹部图像。
