Zhao Chongyue, Zhan Liang, Thompson Paul M, Huang Heng
Department of Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, USA.
Imaging Genetics Center, University of Southern California, Los Angeles, CA, USA.
Med Image Comput Comput Assist Interv. 2022 Sep;13431:356-365. doi: 10.1007/978-3-031-16431-6_34. Epub 2022 Sep 15.
Understanding the intrinsic patterns of human brain is important to make inferences about the mind and brain-behavior association. Electrophysiological methods (i.e. MEG/EEG) provide direct measures of neural activity without the effect of vascular confounds. The blood oxygenated level-dependent (BOLD) signal of functional MRI (fMRI) reveals the spatial and temporal brain activity across different brain regions. However, it is unclear how to associate the high temporal resolution Electrophysiological measures with high spatial resolution fMRI signals. Here, we present a novel interpretable model for coupling the structure and function activity of brain based on heterogeneous contrastive graph representation. The proposed method is able to link manifest variables of the brain (i.e. MEG, MRI, fMRI and behavior performance) and quantify the intrinsic coupling strength of different modal signals. The proposed method learns the heterogeneous node and graph representations by contrasting the structural and temporal views through the mind to multimodal brain data. The first experiment with 1200 subjects from Human connectome Project (HCP) shows that the proposed method outperforms the existing approaches in predicting individual gender and enabling the location of the importance of brain regions with sex difference. The second experiment associates the structure and temporal views between the low-level sensory regions and high-level cognitive ones. The experimental results demonstrate that the dependence of structural and temporal views varied spatially through different modal variants. The proposed method enables the heterogeneous biomarkers explanation for different brain measurements.
了解人类大脑的内在模式对于推断心理以及脑与行为的关联非常重要。电生理方法（即脑磁图/脑电图）可提供神经活动的直接测量结果，而不受血管干扰的影响。功能磁共振成像（fMRI）的血氧水平依赖（BOLD）信号揭示了不同脑区的时空脑活动。然而，尚不清楚如何将高时间分辨率的电生理测量与高空间分辨率的fMRI信号相关联。在此，我们提出了一种基于异构对比图表示的新颖可解释模型，用于耦合大脑的结构和功能活动。所提出的方法能够链接大脑的明显变量（即脑磁图、磁共振成像、功能磁共振成像和行为表现），并量化不同模态信号的内在耦合强度。该方法通过将结构和时间视图通过心智与多模态脑数据进行对比，学习异构节点和图表示。来自人类连接体项目（HCP）的1200名受试者的第一个实验表明，所提出的方法在预测个体性别以及确定具有性别差异的脑区重要性位置方面优于现有方法。第二个实验将低级感觉区域和高级认知区域之间的结构和时间视图联系起来。实验结果表明，结构和时间视图的依赖性在空间上因不同的模态变体而有所不同。所提出的方法能够对不同的脑测量进行异构生物标志物解释。
