Kumar G Vinodh, Kumar Neeraj, Roy Dipanjan, Banerjee Arpan
Cognitive Brain Lab, National Brain Research Centre, NH 8, Manesar, Gurgaon 122051, India.
Centre of Behavioural and Cognitive Sciences, University of Allahabad, Allahabad 211002, India.
Multisens Res. 2018 Jan 1;31(5):481-500. doi: 10.1163/22134808-00002574.
Visual cues from the speaker's face influence the perception of speech. An example of this influence is demonstrated by the McGurk-effect where illusory (cross-modal) sounds are perceived following presentation of incongruent audio-visual (AV) stimuli. Previous studies report the engagement of specific cortical modules that are spatially distributed during cross-modal perception. However, the limits of the underlying representational space and the cortical network mechanisms remain unclear. In this combined psychophysical and electroencephalography (EEG) study, the participants reported their perception while listening to a set of synchronous and asynchronous incongruent AV stimuli. We identified the neural representation of subjective cross-modal perception at different organizational levels - at specific locations in sensor space and at the level of the large-scale brain network estimated from between-sensor interactions. We identified an enhanced positivity in the event-related potential peak around 300 ms following stimulus onset associated with cross-modal perception. At the spectral level, cross-modal perception involved an overall decrease in power at the frontal and temporal regions at multiple frequency bands and at all AV lags, along with an increased power at the occipital scalp region for synchronous AV stimuli. At the level of large-scale neuronal networks, enhanced functional connectivity at the gamma band involving frontal regions serves as a marker of AV integration. Thus, we report in one single study that segregation of information processing at individual brain locations and integration of information over candidate brain networks underlie multisensory speech perception.
说话者面部的视觉线索会影响言语感知。这种影响的一个例子由麦格克效应所证明，即在呈现不一致的视听（AV）刺激后会感知到虚幻的（跨模态）声音。先前的研究报告了在跨模态感知过程中空间分布的特定皮质模块的参与情况。然而，潜在表征空间的限度以及皮质网络机制仍不清楚。在这项结合了心理物理学和脑电图（EEG）的研究中，参与者在听一组同步和异步不一致的AV刺激时报告他们的感知。我们在不同组织层面确定了主观跨模态感知的神经表征——在传感器空间的特定位置以及从传感器间相互作用估计的大规模脑网络层面。我们发现在刺激开始后约300毫秒的事件相关电位峰值处，与跨模态感知相关的正电位增强。在频谱层面，跨模态感知涉及多个频段以及所有AV延迟情况下额叶和颞叶区域功率的整体下降，同时对于同步AV刺激，枕部头皮区域的功率增加。在大规模神经元网络层面，涉及额叶区域的伽马波段功能连接增强是AV整合的一个标志。因此，我们在一项单一研究中报告，个体脑区信息处理的分离以及候选脑网络上信息的整合是多感官言语感知的基础。
