Lin Wanyun, Lin Weiming, Chen Gang, Zhang Hejun, Gao Qinquan, Huang Yechong, Tong Tong, Du Min
Front Neurosci. 2021 Apr 9;15:646013. doi: 10.3389/fnins.2021.646013. eCollection 2021.
Combining multi-modality data for brain disease diagnosis such as Alzheimer's disease (AD) commonly leads to improved performance than those using a single modality. However, it is still challenging to train a multi-modality model since it is difficult in clinical practice to obtain complete data that includes all modality data. Generally speaking, it is difficult to obtain both magnetic resonance images (MRI) and positron emission tomography (PET) images of a single patient. PET is expensive and requires the injection of radioactive substances into the patient's body, while MR images are cheaper, safer, and more widely used in practice. Discarding samples without PET data is a common method in previous studies, but the reduction in the number of samples will result in a decrease in model performance. To take advantage of multi-modal complementary information, we first adopt the Reversible Generative Adversarial Network (RevGAN) model to reconstruct the missing data. After that, a 3D convolutional neural network (CNN) classification model with multi-modality input was proposed to perform AD diagnosis. We have evaluated our method on the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, and compared the performance of the proposed method with those using state-of-the-art methods. The experimental results show that the structural and functional information of brain tissue can be mapped well and that the image synthesized by our method is close to the real image. In addition, the use of synthetic data is beneficial for the diagnosis and prediction of Alzheimer's disease, demonstrating the effectiveness of the proposed framework.
结合多模态数据用于脑部疾病诊断，如阿尔茨海默病（AD），通常比使用单一模态能带来更好的性能。然而，训练一个多模态模型仍然具有挑战性，因为在临床实践中很难获得包含所有模态数据的完整数据。一般来说，获取单个患者的磁共振成像（MRI）和正电子发射断层扫描（PET）图像都很困难。PET成本高昂，需要向患者体内注射放射性物质，而MR图像成本较低、更安全且在实践中应用更广泛。在先前的研究中，丢弃没有PET数据的样本是一种常见方法，但样本数量的减少会导致模型性能下降。为了利用多模态互补信息，我们首先采用可逆生成对抗网络（RevGAN）模型来重建缺失数据。之后，提出了一种具有多模态输入的3D卷积神经网络（CNN）分类模型来进行AD诊断。我们在阿尔茨海默病神经影像倡议（ADNI）数据库上评估了我们的方法，并将所提方法的性能与使用最先进方法的性能进行了比较。实验结果表明，脑组织的结构和功能信息能够得到很好的映射，并且我们方法合成的图像接近真实图像。此外，合成数据的使用有利于阿尔茨海默病的诊断和预测，证明了所提框架的有效性。
