Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, ITE 325 B, 1000 Hilltop Circle, Baltimore, MD 21250, USA.
Neuroimage. 2010 May 1;50(4):1438-45. doi: 10.1016/j.neuroimage.2010.01.062. Epub 2010 Jan 25.
Functional magnetic resonance imaging (fMRI) data and electroencephalography (EEG) data provide complementary spatio-temporal information about brain function. Methods to couple the relative strengths of these modalities usually involve two stages: first forming a feature set from each dataset based on one criterion followed by exploration of connections among the features using a second criterion. We propose a data fusion method for simultaneously acquired fMRI and EEG data that combines these steps using a single criterion for finding the cross-modality associations and performing source separation. Using multi-set canonical correlation analysis (M-CCA), we obtain a decomposition of the two modalities, into spatial maps for fMRI data and a corresponding temporal evolution for EEG data, based on trial-to-trial covariation across the two modalities. Additionally, the analysis is performed on data from a group of subjects in order to make group inferences about the covariation across modalities. Being multivariate, the proposed method facilitates the study of brain connectivity along with localization of brain function. M-CCA can be easily extended to incorporate different data types and additional modalities. We demonstrate the promise of the proposed method in finding covarying trial-to-trial amplitude modulations (AMs) in an auditory task involving implicit pattern learning. The results show approximately linear decreasing trends in AMs for both modalities and the corresponding spatial activations occur mainly in motor, frontal, temporal, inferior parietal, and orbito-frontal areas that are linked both to sensory function as well as learning and expectation--all of which match activations related to the presented paradigm.
功能磁共振成像 (fMRI) 数据和脑电图 (EEG) 数据提供了关于大脑功能的互补时空信息。将这些模态的相对优势结合起来的方法通常涉及两个阶段：首先，根据一个标准从每个数据集形成一个特征集，然后使用第二个标准探索特征之间的连接。我们提出了一种用于同时获取 fMRI 和 EEG 数据的融合方法，该方法使用单一标准来寻找跨模态关联并进行源分离，从而将这两个步骤结合起来。使用多集典范相关分析 (M-CCA)，我们根据两种模态的逐试共变，将两种模态分解为 fMRI 数据的空间图和 EEG 数据的相应时间演化。此外，还对一组受试者的数据进行了分析，以便对模态之间的共变进行组推断。由于是多元的，因此所提出的方法有利于研究大脑连接以及大脑功能的本地化。M-CCA 可以很容易地扩展到包含不同的数据类型和附加的模态。我们通过在涉及内隐模式学习的听觉任务中寻找随时间变化的幅度调制 (AM) 的共变，证明了该方法的前景。结果表明，两种模态的 AM 都呈现出近似线性递减的趋势，相应的空间激活主要发生在运动、额、颞、下顶叶和眶额区域，这些区域与感觉功能以及学习和期望都有关联，所有这些都与呈现的范式相关的激活相匹配。
