Faber Sarah, Brown Tanya, Carpentier Sarah, McIntosh A R
University of Toronto, Toronto, ON, Canada.
Simon Fraser University, Burnaby, BC, Canada.
Imaging Neurosci (Camb). 2025 Jan 2;3. doi: 10.1162/imag_a_00413. eCollection 2025.
The human brain is a complex, adaptive system capable of parsing complex stimuli and generating complex behaviour. Understanding how to model and interpret the dynamic relationship between brain, behaviour, and the environment will provide vital information on how the brain responds to real-world stimuli, develops and ages, and adapts to pathology. Modelling together numerous streams of dynamic data, however, presents sizable methodological challenges. In this paper, we present a novel workflow and sample interpretation of a data set incorporating brain, behavioural, and stimulus data from a music listening study. We use hidden Markov modelling (HMM) to extract state time series from continuous high-dimensional EEG and stimulus data, estimate time series variables consistent with HMM from continuous low-dimensional behavioural data, and model the multi-modal data together using partial least squares (PLS). We offer a sample interpretation of the results, including a discussion on the limitations of the currently available tools, and discuss future directions for dynamic multi-modal analysis focusing on naturalistic behaviours.
人类大脑是一个复杂的自适应系统,能够解析复杂的刺激并产生复杂的行为。了解如何对大脑、行为和环境之间的动态关系进行建模和解释,将为大脑如何响应现实世界的刺激、发育和老化以及适应病理状况提供至关重要的信息。然而,对众多动态数据流进行联合建模带来了相当大的方法学挑战。在本文中,我们展示了一种新颖的工作流程以及对一个数据集的示例解读,该数据集包含来自一项音乐聆听研究的大脑、行为和刺激数据。我们使用隐马尔可夫模型(HMM)从连续的高维脑电图和刺激数据中提取状态时间序列,从连续的低维行为数据中估计与HMM一致的时间序列变量,并使用偏最小二乘法(PLS)对多模态数据进行联合建模。我们提供了结果的示例解读,包括对当前可用工具局限性的讨论,并探讨了专注于自然行为的动态多模态分析的未来方向。