Zhou Meng, Wagner Matthias W, Tabori Uri, Hawkins Cynthia, Ertl-Wagner Birgit B, Khalvati Farzad
Department of Computer Science, University of Toronto, 40 St George St., Toronto, M5S 2E4, ON, Canada; Neurosciences & Mental Health Research Program, The Hospital for Sick Children, 686 Bay St., Toronto, M5G 0A4, ON, Canada.
Department of Diagnostic and Interventional Radiology, The Hospital for Sick Children, 170 Elizabeth St., Toronto, M5G 1H3, ON, Canada; Institute of Diagnostic and Interventional Neuroradiology, University Hospital Augsburg, Stenglinstraße 2, Augsburg, 86156, Germany.
Comput Biol Med. 2025 Feb;185:109502. doi: 10.1016/j.compbiomed.2024.109502. Epub 2024 Dec 19.
Medical image analysis has significantly benefited from advancements in deep learning, particularly in the application of Generative Adversarial Networks (GANs) for generating realistic and diverse images that can augment training datasets. The common GAN-based approach is to generate entire image volumes, rather than the region of interest (ROI). Research on deep learning-based brain tumor classification using MRI has shown that it is easier to classify the tumor ROIs compared to the entire image volumes. In this work, we present a novel framework that uses vector-quantization GAN and a transformer incorporating masked token modeling to generate high-resolution and diverse 3D brain tumor ROIs that can be used as additional data for tumor ROI classification. We apply our method to two imbalanced datasets where we augment the minority class: (1) low-grade glioma (LGG) ROIs from the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2019 dataset; (2) BRAF V600E Mutation genetic marker tumor ROIs from the internal pediatric LGG (pLGG) dataset. We show that the proposed method outperforms various baseline models qualitatively and quantitatively. The generated data was used to balance the data to classify brain tumor types. Our approach demonstrates superior performance, surpassing baseline models by 6.4% in the area under the ROC curve (AUC) on the BraTS 2019 dataset and 4.3% in the AUC on the internal pLGG dataset. The results indicate the generated tumor ROIs can effectively address the imbalanced data problem. Our proposed method has the potential to facilitate an accurate diagnosis of rare brain tumors using MRI scans.
医学图像分析从深度学习的进步中受益匪浅，特别是在生成对抗网络（GAN）用于生成逼真且多样的图像以扩充训练数据集方面。基于GAN的常见方法是生成整个图像体积，而不是感兴趣区域（ROI）。关于使用MRI进行基于深度学习的脑肿瘤分类的研究表明，与整个图像体积相比，对肿瘤ROI进行分类更容易。在这项工作中，我们提出了一个新颖的框架，该框架使用矢量量化GAN和结合掩码令牌建模的变压器来生成高分辨率和多样的3D脑肿瘤ROI，这些ROI可作为肿瘤ROI分类的额外数据。我们将我们的方法应用于两个不平衡数据集，在其中扩充少数类：（1）来自多模态脑肿瘤分割挑战赛（BraTS）2019数据集的低级别胶质瘤（LGG）ROI；（2）来自内部儿科LGG（pLGG）数据集的BRAF V600E突变遗传标记肿瘤ROI。我们表明，所提出的方法在定性和定量方面均优于各种基线模型。生成的数据用于平衡数据以对脑肿瘤类型进行分类。我们的方法表现出卓越的性能，在BraTS 2019数据集上的ROC曲线下面积（AUC）比基线模型高出6.4%，在内部pLGG数据集上的AUC高出4.3%。结果表明，生成的肿瘤ROI可以有效地解决数据不平衡问题。我们提出的方法有可能促进使用MRI扫描对罕见脑肿瘤进行准确诊断。
