Xiong Xiangyu, Sun Yue, Liu Xiaohong, Ke Wei, Lam Chan-Tong, Gao Qinquan, Tong Tong, Li Shuo, Tan Tao
Faculty of Applied Sciences, Macao Polytechnic University, Macao, 999078, China.
John Hopcroft Center (JHC) for Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China.
Comput Methods Programs Biomed. 2025 Nov;271:109016. doi: 10.1016/j.cmpb.2025.109016. Epub 2025 Aug 16.
Modern deep neural networks are highly over-parameterized, necessitating the use of data augmentation techniques to prevent overfitting and enhance generalization. Generative adversarial networks (GANs) are popular for synthesizing visually realistic images. However, these synthetic images often lack diversity and may have ambiguous class labels. Recent data mixing strategies address some of these issues by mixing image labels based on salient regions. Since the main diagnostic information is not always contained within the salient regions, we aim to address the resulting label mismatches in medical image classifications.
We propose a variety-guided data mixing framework (VariMix), which exploits an absolute difference map (ADM) to address the label mismatch problems of mixed medical images. VariMix generates ADM using the image-to-image (I2I) GAN across multiple classes and allows for bidirectional mixing operations between the training samples.
The proposed VariMix achieves the highest accuracy of 99.30% and 94.60% with a SwinT V2 classifier on a Chest X-ray (CXR) dataset and a Retinal dataset, respectively. It also achieves the highest accuracy of 87.73%, 99.28%, 95.13%, and 95.81% with a ConvNeXt classifier on a Breast Ultrasound (US) dataset, a CXR dataset, a Retinal dataset, and a Maternal-Fetal US dataset, respectively. Furthermore, the medical expert evaluation on generated images shows the great potential of our proposed I2I GAN in improving the accuracy of medical image classifications.
Extensive experiments demonstrate the superiority of VariMix compared with the existing GAN- and Mixup-based methods on four public datasets using Swin Transformer V2 and ConvNeXt architectures. Furthermore, by projecting the source image to the hyperplanes of the classifiers, the proposed I2I GAN can generate hyperplane difference maps between the source image and the hyperplane image, demonstrating its ability to interpret medical image classifications. The source code is provided in https://github.com/yXiangXiong/VariMix.
现代深度神经网络具有高度的过参数化特性，因此需要使用数据增强技术来防止过拟合并提高泛化能力。生成对抗网络（GAN）在合成视觉上逼真的图像方面很受欢迎。然而，这些合成图像往往缺乏多样性，并且可能具有模糊的类别标签。最近的数据混合策略通过基于显著区域混合图像标签来解决其中一些问题。由于主要诊断信息并不总是包含在显著区域内，我们旨在解决医学图像分类中由此产生的标签不匹配问题。
我们提出了一种多类别引导的数据混合框架（VariMix），该框架利用绝对差异图（ADM）来解决混合医学图像的标签不匹配问题。VariMix使用跨多个类别的图像到图像（I2I）GAN生成ADM，并允许在训练样本之间进行双向混合操作。
所提出的VariMix在胸部X光（CXR）数据集和视网膜数据集上，使用SwinT V2分类器分别达到了99.30%和94.60%的最高准确率。在乳腺超声（US）数据集、CXR数据集、视网膜数据集和母胎US数据集上，使用ConvNeXt分类器分别达到了87.73%、99.28%、95.13%和95.81%的最高准确率。此外，医学专家对生成图像的评估表明，我们提出的I2I GAN在提高医学图像分类准确率方面具有巨大潜力。
大量实验表明，在使用Swin Transformer V2和ConvNeXt架构的四个公共数据集上，VariMix与现有的基于GAN和Mixup的方法相比具有优越性。此外，通过将源图像投影到分类器的超平面上，所提出的I2I GAN可以生成源图像与超平面图像之间的超平面差异图，证明了其解释医学图像分类的能力。源代码可在https://github.com/yXiangXiong/VariMix获取。
