Ding Jun-En, Hsu Chien-Chin, Liu Feng
School of Systems and Enterprises, Stevens Institute of Technology, Hoboken, NJ 07030, USA.
Dept. Nuclear Medicine, Kaohsiung Chang Gung Memorial Hospital, Kaohsiung Chang, Taiwan.
Proc IEEE Int Symp Biomed Imaging. 2024 May;2024. doi: 10.1109/isbi56570.2024.10635712. Epub 2024 Aug 22.
Parkinson's Disease (PD) affects millions globally, impacting movement. Prior research utilized deep learning for PD prediction, primarily focusing on medical images, neglecting the data's underlying manifold structure. This work proposes a multimodal approach encompassing both image and non-image features, leveraging contrastive cross-view graph fusion for PD classification. We introduce a novel multimodal co-attention module, integrating embeddings from separate graph views derived from low-dimensional representations of images and clinical features. This enables more robust and structured feature extraction for improved multi-view data analysis. Additionally, a simplified contrastive loss-based fusion method is devised to enhance cross-view fusion learning. Our graph-view multimodal approach achieves an accuracy of 91% and an area under the receiver operating characteristic curve (AUC) of 92.8% in five-fold cross-validation. It also demonstrates superior predictive capabilities on non-image data compared to solely machine learning-based methods.
帕金森病（PD）在全球影响着数百万人，对运动产生影响。先前的研究利用深度学习进行PD预测，主要集中在医学图像上，而忽略了数据潜在的流形结构。这项工作提出了一种多模态方法，涵盖图像和非图像特征，利用对比跨视图图融合进行PD分类。我们引入了一种新颖的多模态协同注意力模块，整合了来自图像和临床特征的低维表示所衍生的单独图视图的嵌入。这使得能够进行更强大且结构化的特征提取，以改进多视图数据分析。此外，还设计了一种基于简化对比损失的融合方法，以增强跨视图融合学习。我们的图视图多模态方法在五折交叉验证中实现了91%的准确率和92.8%的受试者工作特征曲线下面积（AUC）。与仅基于机器学习的方法相比，它在非图像数据上也表现出卓越的预测能力。
