Centre for Cognitive and Brain Sciences, University of Macau, Taipa, China.
Faculty of Arts and Humanities, University of Macau, Taipa, China.
Hum Brain Mapp. 2024 Feb 1;45(2):e26617. doi: 10.1002/hbm.26617.
Natural language processing unfolds information overtime as spatially separated, multimodal, and interconnected neural processes. Existing noninvasive subtraction-based neuroimaging techniques cannot simultaneously achieve the spatial and temporal resolutions required to visualize ongoing information flows across the whole brain. Here we have developed rapid phase-encoded designs to fully exploit the temporal information latent in functional magnetic resonance imaging data, as well as overcoming scanner noise and head-motion challenges during overt language tasks. We captured real-time information flows as coherent hemodynamic waves traveling over the cortical surface during listening, reading aloud, reciting, and oral cross-language interpreting tasks. We were able to observe the timing, location, direction, and surge of traveling waves in all language tasks, which were visualized as "brainstorms" on brain "weather" maps. The paths of hemodynamic traveling waves provide direct evidence for dual-stream models of the visual and auditory systems as well as logistics models for crossmodal and cross-language processing. Specifically, we have tracked down the step-by-step processing of written or spoken sentences first being received and processed by the visual or auditory streams, carried across language and domain-general cognitive regions, and finally delivered as overt speeches monitored through the auditory cortex, which gives a complete picture of information flows across the brain during natural language functioning. PRACTITIONER POINTS: Phase-encoded fMRI enables simultaneous imaging of high spatial and temporal resolution, capturing continuous spatiotemporal dynamics of the entire brain during real-time overt natural language tasks. Spatiotemporal traveling wave patterns provide direct evidence for constructing comprehensive and explicit models of human information processing. This study unlocks the potential of applying rapid phase-encoded fMRI to indirectly track the underlying neural information flows of sequential sensory, motor, and high-order cognitive processes.
自然语言处理随着时间的推移展开信息,这些信息是空间上分离的、多模态的和相互关联的神经过程。现有的非侵入性基于减法的神经影像学技术无法同时实现可视化整个大脑中持续信息流所需的空间和时间分辨率。在这里,我们开发了快速相位编码设计,以充分利用功能磁共振成像数据中潜在的时间信息,以及克服在显性语言任务期间的扫描仪噪声和头部运动挑战。我们在听、大声朗读、背诵和口头跨语言口译任务期间,捕获了沿皮质表面传播的连贯血液动力学波作为实时信息流。我们能够观察到所有语言任务中传播波的时间、位置、方向和涌动,这些波在大脑“天气”图上被可视化为“脑暴”。血液动力学传播波的路径为视觉和听觉系统的双流模型以及跨模态和跨语言处理的物流模型提供了直接证据。具体来说,我们追踪了书面或口头句子的逐步处理,这些句子首先被视觉或听觉流接收和处理,跨越语言和领域通用认知区域,最后通过听觉皮层作为显性言语传递,这为自然语言功能期间大脑内信息流提供了完整的画面。实践者要点:相位编码 fMRI 能够同时进行高空间和高时间分辨率成像,在实时显性自然语言任务期间捕获整个大脑的连续时空动态。时空传播波模式为构建人类信息处理的综合和明确模型提供了直接证据。这项研究解锁了应用快速相位编码 fMRI 来间接跟踪连续感觉、运动和高阶认知过程的潜在神经信息流的潜力。