Quantitative Biomedical Research Center, Peter O'Donnell Jr. School of Public Health, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
Department of Pathology, Division of Pathology/Lab Medicine, The University of Texas MD, Anderson Cancer Center, Houston, TX 77030, USA.
Comput Methods Programs Biomed. 2023 Nov;241:107768. doi: 10.1016/j.cmpb.2023.107768. Epub 2023 Aug 19.
Unsupervised domain adaptation (UDA) is a powerful approach in tackling domain discrepancies and reducing the burden of laborious and error-prone pixel-level annotations for instance segmentation. However, the domain adaptation strategies utilized in previous instance segmentation models pool all the labeled/detected instances together to train the instance-level GAN discriminator, which neglects the differences among multiple instance categories. Such pooling prevents UDA instance segmentation models from learning categorical correspondence between source and target domains for accurate instance classification; METHODS: To tackle this challenge, we propose an Instance Segmentation CycleGAN (ISC-GAN) algorithm for UDA multiclass-instance segmentation. We conduct extensive experiments on the multiclass nuclei recognition task to transfer knowledge from hematoxylin and eosin to immunohistochemistry stained pathology images. Specifically, we fuse CycleGAN with Mask R-CNN to learn categorical correspondence with image-level domain adaptation and virtual supervision. Moreover, we utilize Curriculum Learning to separate the learning process into two steps: (1) learning segmentation only on labeled source data, and (2) learning target domain segmentation with paired virtual labels generated by ISC-GAN. The performance was further improved through experiments with other strategies, including Shared Weights, Knowledge Distillation, and Expanded Source Data.
Comparing to the baseline model or the three UDA instance detection and segmentation models, ISC-GAN illustrates the state-of-the-art performance, with 39.1% average precision and 48.7% average recall. The source codes of ISC-GAN are available at https://github.com/sdw95927/InstanceSegmentation-CycleGAN.
ISC-GAN adapted knowledge from hematoxylin and eosin to immunohistochemistry stained pathology images, suggesting the potential for reducing the need for large annotated pathological image datasets in deep learning and computer vision tasks.
无监督领域自适应（UDA）是解决领域差异和减少实例分割中费力且容易出错的像素级注释负担的强大方法。然而，以前的实例分割模型中使用的领域自适应策略将所有标记/检测到的实例都合并在一起，以训练实例级 GAN 判别器，这忽略了多个实例类之间的差异。这种合并阻止了 UDA 实例分割模型学习源域和目标域之间的类别对应关系，从而无法进行准确的实例分类；方法：为了解决这个挑战，我们提出了一种用于 UDA 多类别实例分割的实例分割 CycleGAN（ISC-GAN）算法。我们在多类别细胞核识别任务上进行了广泛的实验，以从苏木精和伊红转移知识到免疫组织化学染色的病理学图像。具体来说，我们将 CycleGAN 与 Mask R-CNN 融合，通过图像级别的领域自适应和虚拟监督来学习类别对应关系。此外，我们利用课程学习将学习过程分为两个步骤：（1）仅在标记的源数据上学习分割，（2）使用 ISC-GAN 生成的虚拟配对标签学习目标域分割。通过实验还采用了其他策略，包括共享权重、知识蒸馏和扩展源数据，进一步提高了性能。结果：与基线模型或三种 UDA 实例检测和分割模型相比，ISC-GAN 展示了最先进的性能，平均精度为 39.1%，平均召回率为 48.7%。ISC-GAN 的源代码可在 https://github.com/sdw95927/InstanceSegmentation-CycleGAN 上获得。结论：ISC-GAN 从苏木精和伊红转移知识到免疫组织化学染色的病理学图像，这表明在深度学习和计算机视觉任务中减少对大型注释病理学图像数据集的需求具有潜力。
