Shirae Satoshi, Debsarkar Shyam Sundar, Kawanaka Hiroharu, Aronow Bruce, Prasath V B Surya
Graduate School of Engineering, Mie University, Tsu, Mie 514-8507, Japan.
Department of Computer Science, University of Cincinnati, Cincinnati, OH 45221, USA.
IEEE Access. 2025;13:57780-57797. doi: 10.1109/access.2025.3556713. Epub 2025 Apr 1.
Glioma is the most common malignant tumor of the central nervous system, and diffuse Glioma is classified as grades II-IV by world health organization (WHO). In the the cancer genome atlas (TCGA) Glioma dataset, grade II and III Gliomas are classified as low-grade glioma (LGG), and grade IV Gliomas as glioblastoma multiforme (GBM). In clinical practice, the survival and treatment process with Glioma patients depends on properly diagnosing the subtype. With this background, there has been much research on Glioma over the years. Among these researches, the origin and evolution of whole slide images (WSIs) have led to many attempts to support diagnosis by image analysis. On the other hand, due to the disease complexities of Glioma patients, multimodal analysis using various types of data rather than a single data set has been attracting attention. In our proposed method, multiple deep learning models are used to extract features from histopathology images, and the features of the obtained images are concatenated with those of the clinical data in a fusion approach. Then, we perform patch-level classification by machine learning (ML) using the concatenated features. Based on the performances of the deep learning models, we ensemble feature sets from top three models and perform further classifications. In the experiments with our proposed ensemble fusion AI (EFAI) approach using WSIs and clinical data of diffuse Glioma patients on TCGA dataset, the classification accuracy of the proposed multimodal ensemble fusion method is 0.936 with an area under the curve (AUC) value of 0.967 when tested on a balanced dataset of 240 GBM, 240 LGG patients. On an imbalanced dataset of 141 GBM, 242 LGG patients the proposed method obtained the accuracy of 0.936 and AUC of 0.967. Our proposed ensemble fusion approach significantly outperforms the classification using only histopathology images alone with deep learning models. Therefore, our approach can be used to support the diagnosis of Glioma patients and can lead to better diagnosis.
胶质瘤是中枢神经系统最常见的恶性肿瘤，世界卫生组织（WHO）将弥漫性胶质瘤分为II - IV级。在癌症基因组图谱（TCGA）胶质瘤数据集中，II级和III级胶质瘤被归类为低级别胶质瘤（LGG），IV级胶质瘤为多形性胶质母细胞瘤（GBM）。在临床实践中，胶质瘤患者的生存和治疗过程取决于正确诊断亚型。在此背景下，多年来对胶质瘤进行了大量研究。在这些研究中，全切片图像（WSIs）的起源和演变引发了许多通过图像分析辅助诊断的尝试。另一方面，由于胶质瘤患者疾病的复杂性，使用各种类型数据而非单一数据集的多模态分析受到关注。在我们提出的方法中，使用多个深度学习模型从组织病理学图像中提取特征，并通过融合方法将获得图像的特征与临床数据的特征连接起来。然后，我们使用连接后的特征通过机器学习（ML）进行切片级分类。基于深度学习模型的性能，我们整合前三个模型的特征集并进行进一步分类。在使用TCGA数据集上弥漫性胶质瘤患者的WSIs和临床数据进行的我们提出的集成融合人工智能（EFAI）方法实验中，当在240例GBM、240例LGG患者的平衡数据集上进行测试时，所提出的多模态集成融合方法的分类准确率为0.936，曲线下面积（AUC）值为0.967。在141例GBM、242例LGG患者的不平衡数据集上，所提出的方法获得了0.936的准确率和0.967的AUC。我们提出的集成融合方法显著优于仅使用深度学习模型的组织病理学图像进行的分类。因此，我们的方法可用于辅助胶质瘤患者的诊断，并能实现更好的诊断。
