IEEE Trans Biomed Eng. 2023 Oct;70(10):2822-2833. doi: 10.1109/TBME.2023.3265679. Epub 2023 Sep 27.
Accurate visual classification of bladder tissue during Trans-Urethral Resection of Bladder Tumor (TURBT) procedures is essential to improve early cancer diagnosis and treatment. During TURBT interventions, White Light Imaging (WLI) and Narrow Band Imaging (NBI) techniques are used for lesion detection. Each imaging technique provides diverse visual information that allows clinicians to identify and classify cancerous lesions. Computer vision methods that use both imaging techniques could improve endoscopic diagnosis. We address the challenge of tissue classification when annotations are available only in one domain, in our case WLI, and the endoscopic images correspond to an unpaired dataset, i.e. there is no exact equivalent for every image in both NBI and WLI domains.
We propose a semi-surprised Generative Adversarial Network (GAN)-based method composed of three main components: a teacher network trained on the labeled WLI data; a cycle-consistency GAN to perform unpaired image-to-image translation, and a multi-input student network. To ensure the quality of the synthetic images generated by the proposed GAN we perform a detailed quantitative, and qualitative analysis with the help of specialists.
The overall average classification accuracy, precision, and recall obtained with the proposed method for tissue classification are 0.90, 0.88, and 0.89 respectively, while the same metrics obtained in the unlabeled domain (NBI) are 0.92, 0.64, and 0.94 respectively. The quality of the generated images is reliable enough to deceive specialists.
This study shows the potential of using semi-supervised GAN-based bladder tissue classification when annotations are limited in multi-domain data.
在经尿道膀胱肿瘤切除术（TURBT）过程中，对膀胱组织进行准确的视觉分类对于提高早期癌症诊断和治疗至关重要。在 TURBT 干预过程中，使用白光成像（WLI）和窄带成像（NBI）技术进行病变检测。每种成像技术都提供了不同的视觉信息，使临床医生能够识别和分类癌性病变。使用这两种成像技术的计算机视觉方法可以改善内窥镜诊断。当注释仅在一个域（在我们的案例中为 WLI）中可用，并且内窥镜图像对应于未配对数据集时，我们解决了组织分类的挑战，即 NBI 和 WLI 域中没有每个图像的确切对应。
我们提出了一种基于半惊喜生成对抗网络（GAN）的方法，该方法由三个主要组件组成：在标记的 WLI 数据上训练的教师网络；执行未配对图像到图像转换的循环一致性 GAN，以及多输入学生网络。为了确保所提出的 GAN 生成的合成图像的质量，我们在专家的帮助下进行了详细的定量和定性分析。
在所提出的方法中，用于组织分类的总体平均分类准确率、精度和召回率分别为 0.90、0.88 和 0.89，而在未标记域（NBI）中获得的相同指标分别为 0.92、0.64 和 0.94。生成图像的质量可靠到足以欺骗专家。
这项研究表明，在多域数据中注释有限的情况下，使用基于半监督 GAN 的膀胱组织分类具有潜力。
