Rotman Research Institute, Baycrest Centre, Toronto, Canada.
J Neurophysiol. 2011 Dec;106(6):2896-909. doi: 10.1152/jn.00303.2011. Epub 2011 Aug 31.
Perceptual objects often comprise a visual and auditory signature that arrives simultaneously through distinct sensory channels, and cross-modal features are linked by virtue of being attributed to a specific object. Continued exposure to cross-modal events sets up expectations about what a given object most likely "sounds" like, and vice versa, thereby facilitating object detection and recognition. The binding of familiar auditory and visual signatures is referred to as semantic, multisensory integration. Whereas integration of semantically related cross-modal features is behaviorally advantageous, situations of sensory dominance of one modality at the expense of another impair performance. In the present study, magnetoencephalography recordings of semantically related cross-modal and unimodal stimuli captured the spatiotemporal patterns underlying multisensory processing at multiple stages. At early stages, 100 ms after stimulus onset, posterior parietal brain regions responded preferentially to cross-modal stimuli irrespective of task instructions or the degree of semantic relatedness between the auditory and visual components. As participants were required to classify cross-modal stimuli into semantic categories, activity in superior temporal and posterior cingulate cortices increased between 200 and 400 ms. As task instructions changed to incorporate cross-modal conflict, a process whereby auditory and visual components of cross-modal stimuli were compared to estimate their degree of congruence, multisensory processes were captured in parahippocampal, dorsomedial, and orbitofrontal cortices 100 and 400 ms after stimulus onset. Our results suggest that multisensory facilitation is associated with posterior parietal activity as early as 100 ms after stimulus onset. However, as participants are required to evaluate cross-modal stimuli based on their semantic category or their degree of congruence, multisensory processes extend in cingulate, temporal, and prefrontal cortices.
知觉对象通常由通过不同感觉通道同时到达的视觉和听觉特征组成，跨模态特征通过归因于特定对象而联系在一起。持续接触跨模态事件会对给定对象最有可能“听起来”的样子产生期望，反之亦然，从而促进对象检测和识别。熟悉的听觉和视觉特征的绑定称为语义、多感觉整合。虽然语义相关的跨模态特征的整合在行为上是有利的，但一种模态对另一种模态的感觉优势会损害表现。在本研究中，通过语义相关的跨模态和单模态刺激的脑磁图记录，捕获了多感觉处理在多个阶段的时空模式。在早期，即刺激开始后 100 毫秒，后顶叶脑区对跨模态刺激有优先反应，而不受任务指令或听觉和视觉成分之间的语义相关性的影响。由于要求参与者将跨模态刺激分类为语义类别，颞上和后扣带回皮质的活动在 200 到 400 毫秒之间增加。随着任务指令的改变，纳入跨模态冲突，即通过比较跨模态刺激的听觉和视觉成分来估计它们的一致性程度，在刺激开始后 100 和 400 毫秒，海马旁回、背内侧和眶额皮质中捕获了多感觉过程。我们的结果表明，多感觉促进与刺激开始后 100 毫秒后的后顶叶活动有关。然而，由于要求参与者根据其语义类别或其一致性程度评估跨模态刺激，多感觉过程在扣带回、颞叶和前额叶皮质中扩展。
