The Frederick J. and Marion A. Schindler Cognitive Neurophysiology Laboratory, The Ernest J. Del Monte Institute for Neuroscience, Department of Neuroscience, University of Rochester School of Medicine and Dentistry, Rochester, New York, 14642, USA; Department of Imaging Sciences, University of Rochester Medical Center, University of Rochester School of Medicine and Dentistry, Rochester, New York, 14642, USA; The Cognitive Neurophysiology Laboratory, Departments of Pediatrics and Neuroscience, Albert Einstein College of Medicine & Montefiore Medical Center, Bronx, New York, 10461, USA.
The Frederick J. and Marion A. Schindler Cognitive Neurophysiology Laboratory, The Ernest J. Del Monte Institute for Neuroscience, Department of Neuroscience, University of Rochester School of Medicine and Dentistry, Rochester, New York, 14642, USA; The Cognitive Neurophysiology Laboratory, Departments of Pediatrics and Neuroscience, Albert Einstein College of Medicine & Montefiore Medical Center, Bronx, New York, 10461, USA.
Neuroimage. 2022 Nov;263:119598. doi: 10.1016/j.neuroimage.2022.119598. Epub 2022 Aug 30.
This fMRI study investigated the effect of seeing articulatory movements of a speaker while listening to a naturalistic narrative stimulus. It had the goal to identify regions of the language network showing multisensory enhancement under synchronous audiovisual conditions. We expected this enhancement to emerge in regions known to underlie the integration of auditory and visual information such as the posterior superior temporal gyrus as well as parts of the broader language network, including the semantic system. To this end we presented 53 participants with a continuous narration of a story in auditory alone, visual alone, and both synchronous and asynchronous audiovisual speech conditions while recording brain activity using BOLD fMRI. We found multisensory enhancement in an extensive network of regions underlying multisensory integration and parts of the semantic network as well as extralinguistic regions not usually associated with multisensory integration, namely the primary visual cortex and the bilateral amygdala. Analysis also revealed involvement of thalamic brain regions along the visual and auditory pathways more commonly associated with early sensory processing. We conclude that under natural listening conditions, multisensory enhancement not only involves sites of multisensory integration but many regions of the wider semantic network and includes regions associated with extralinguistic sensory, perceptual and cognitive processing.
这项 fMRI 研究调查了在聆听自然叙事刺激时观察说话者发音动作对听者的影响。其目的是确定在同步视听条件下表现出多感觉增强的语言网络区域。我们预计这种增强会出现在已知的听觉和视觉信息整合的区域中，如后上颞叶，以及更广泛的语言网络的部分区域，包括语义系统。为此，我们向 53 名参与者展示了一个故事的连续叙述，参与者在单独聆听音频、单独观看视频以及同步和异步视听语音条件下，同时使用 BOLD fMRI 记录大脑活动。我们发现，在多感觉整合的广泛网络区域以及语义网络的部分区域以及通常与多感觉整合无关的语言外区域中，存在多感觉增强，即初级视觉皮层和双侧杏仁核。分析还揭示了与视觉和听觉通路相关的丘脑脑区域的参与，这些区域通常与早期感觉处理有关。我们得出结论，在自然聆听条件下，多感觉增强不仅涉及多感觉整合的部位，而且涉及更广泛的语义网络的许多区域，包括与语言外的感觉、知觉和认知处理相关的区域。
