Liu Baolin, Wang Zhongning, Jin Zhixing
Department of Computer Science and Technology, Tsinghua University, Beijing 100084, PR China.
Neurosci Lett. 2009 Sep 11;461(1):7-11. doi: 10.1016/j.neulet.2009.05.082. Epub 2009 Jun 9.
In real life, the human brain usually receives information through visual and auditory channels and processes the multisensory information, but studies on the integration processing of the dynamic visual and auditory information are relatively few. In this paper, we have designed an experiment, where through the presentation of common scenario, real-world videos, with matched and mismatched actions (images) and sounds as stimuli, we aimed to study the integration processing of synchronized visual and auditory information in videos of real-world events in the human brain, through the use event-related potentials (ERPs) methods. Experimental results showed that videos of mismatched actions (images) and sounds would elicit a larger P400 as compared to videos of matched actions (images) and sounds. We believe that the P400 waveform might be related to the cognitive integration processing of mismatched multisensory information in the human brain. The results also indicated that synchronized multisensory information would interfere with each other, which would influence the results of the cognitive integration processing.
在现实生活中，人类大脑通常通过视觉和听觉通道接收信息并处理多感官信息，但关于动态视觉和听觉信息整合处理的研究相对较少。在本文中，我们设计了一项实验，通过呈现常见场景、真实世界视频，并以匹配和不匹配的动作（图像）与声音作为刺激，旨在利用事件相关电位（ERP）方法研究人类大脑中真实世界事件视频中同步视觉和听觉信息的整合处理。实验结果表明，与动作（图像）和声音匹配的视频相比，动作（图像）和声音不匹配的视频会引发更大的P400。我们认为，P400波形可能与人脑不匹配多感官信息的认知整合处理有关。结果还表明，同步的多感官信息会相互干扰，这会影响认知整合处理的结果。
