Brainnetome Center and National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China.
Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, GA, USA.
Med Image Anal. 2022 May;78:102413. doi: 10.1016/j.media.2022.102413. Epub 2022 Mar 2.
Functional magnetic resonance imaging (fMRI) as a promising tool to investigate psychotic disorders can be decomposed into useful imaging features such as time courses (TCs) of independent components (ICs) and functional network connectivity (FNC) calculated by TC cross-correlation. TCs reflect the temporal dynamics of brain activity and the FNC characterizes temporal coherence across intrinsic brain networks. Both features have been used as input to deep learning approaches with decent results. However, few studies have tried to leverage their complementary information to learn optimal representations at multiple facets. Motivated by this, we proposed a Hybrid Deep Learning Framework integrating brain Connectivity and Activity (HDLFCA) together by combining convolutional recurrent neural network (C-RNN) and deep neural network (DNN), aiming to improve classification accuracy and interpretability simultaneously. Specifically, C-RNN was proposed to extract temporal dynamic dependencies with an attention module (AM) to automatically learn discriminative knowledge from TC nodes, while DNN was applied to identify the most group-discriminative FNC patterns with layer-wise relevance propagation (LRP). Then, both prediction outputs were concatenated to build a new feature matrix, generating the final decision by logistic regression. The effectiveness of HDLFCA was validated on both multi-site schizophrenia (SZ, n ∼ 1100) and public autism datasets (ABIDE, n ∼ 1522) by outperforming 12 alternative models at 2.8-8.9% accuracy, including 8 models using either static FNC or TCs and 4 models using dynamic FNC. Appreciable classification accuracy was achieved for HC vs. SZ (85.3%) and HC vs. Autism (72.4%) respectively. More importantly, the most group-discriminative brain regions can be easily attributed and visualized, providing meaningful biological interpretability and highlighting the great potential of the proposed HDLFCA model in the identification of valid neuroimaging biomarkers.
功能磁共振成像 (fMRI) 作为一种有前途的研究精神障碍的工具，可以分解为有用的成像特征，如独立成分 (IC) 的时间历程 (TC) 和通过 TC 互相关计算的功能网络连接 (FNC)。TC 反映了大脑活动的时间动态，而 FNC 则描述了内在大脑网络之间的时间相干性。这两个特征都已被用于深度学习方法，取得了不错的效果。然而，很少有研究试图利用它们的互补信息来学习多个方面的最佳表示。受此启发，我们提出了一种结合脑连接和活动的混合深度学习框架 (HDLFCA)，通过结合卷积递归神经网络 (C-RNN) 和深度神经网络 (DNN) 来实现这一目标，旨在同时提高分类准确性和可解释性。具体来说，C-RNN 提出了使用注意力模块 (AM) 提取时间动态依赖性，以自动从 TC 节点中学习有区别的知识，而 DNN 则应用于识别最具组判别力的 FNC 模式，并使用层间相关性传播 (LRP)。然后，将两个预测输出连接在一起，构建一个新的特征矩阵，通过逻辑回归生成最终决策。HDLFCA 在多站点精神分裂症 (SZ，n∼1100) 和公共自闭症数据集 (ABIDE，n∼1522) 上进行了验证，在 2.8-8.9%的精度上优于 12 个替代模型，包括 8 个使用静态 FNC 或 TC 的模型和 4 个使用动态 FNC 的模型。在 HC 与 SZ (85.3%)和 HC 与自闭症 (72.4%)的分类中，都取得了可接受的分类精度。更重要的是，最具组判别力的大脑区域可以很容易地归因和可视化，提供了有意义的生物学可解释性，并突出了所提出的 HDLFCA 模型在识别有效神经影像学生物标志物方面的巨大潜力。
