Max Planck Institute for Biological Cybernetics, 72076 Tübingen, Germany.
J Neurosci. 2010 Feb 17;30(7):2662-75. doi: 10.1523/JNEUROSCI.5091-09.2010.
Multisensory interactions have been demonstrated in a distributed neural system encompassing primary sensory and higher-order association areas. However, their distinct functional roles in multisensory integration remain unclear. This functional magnetic resonance imaging study dissociated the functional contributions of three cortical levels to multisensory integration in object categorization. Subjects actively categorized or passively perceived noisy auditory and visual signals emanating from everyday actions with objects. The experiment included two 2 x 2 factorial designs that manipulated either (1) the presence/absence or (2) the informativeness of the sensory inputs. These experimental manipulations revealed three patterns of audiovisual interactions. (1) In primary auditory cortices (PACs), a concurrent visual input increased the stimulus salience by amplifying the auditory response regardless of task-context. Effective connectivity analyses demonstrated that this automatic response amplification is mediated via both direct and indirect [via superior temporal sulcus (STS)] connectivity to visual cortices. (2) In STS and intraparietal sulcus (IPS), audiovisual interactions sustained the integration of higher-order object features and predicted subjects' audiovisual benefits in object categorization. (3) In the left ventrolateral prefrontal cortex (vlPFC), explicit semantic categorization resulted in suppressive audiovisual interactions as an index for multisensory facilitation of semantic retrieval and response selection. In conclusion, multisensory integration emerges at multiple processing stages within the cortical hierarchy. The distinct profiles of audiovisual interactions dissociate audiovisual salience effects in PACs, formation of object representations in STS/IPS and audiovisual facilitation of semantic categorization in vlPFC. Furthermore, in STS/IPS, the profiles of audiovisual interactions were behaviorally relevant and predicted subjects' multisensory benefits in performance accuracy.
多感觉交互作用已在包括主要感觉和高级联想区域的分布式神经系统中得到证实。然而，它们在多感觉整合中的独特功能作用尚不清楚。本功能磁共振成像研究将三个皮质水平的功能贡献与物体分类中的多感觉整合区分开来。受试者主动或被动感知来自日常动作与物体的嘈杂听觉和视觉信号。实验包括两个 2x2 因子设计，分别操纵（1）感觉输入的存在/不存在或（2）感觉输入的信息量。这些实验操作揭示了三种视听相互作用模式。（1）在初级听觉皮层（PAC）中，即使在任务背景下，同时出现的视觉输入也会通过放大听觉反应来增加刺激的显著性。有效连通性分析表明，这种自动反应放大是通过直接和间接（通过颞上沟（STS））与视觉皮层的连通来介导的。（2）在 STS 和顶内沟（IPS）中，视听相互作用持续整合高级物体特征，并预测了受试者在物体分类中的视听收益。（3）在左侧腹外侧前额叶皮层（vlPFC）中，明确的语义分类导致抑制性视听相互作用，作为语义检索和反应选择的多感觉促进的指标。总之，多感觉整合在皮质层次结构内的多个处理阶段出现。视听相互作用的独特模式将 PAC 中的视听显著性效应、STS/IPS 中的物体表示形成以及 vlPFC 中的语义分类的视听促进区分开来。此外，在 STS/IPS 中，视听相互作用的模式与行为相关，并预测了受试者在性能准确性方面的多感觉收益。
