Möttönen Riikka, Krause Christina M, Tiippana Kaisa, Sams Mikko
Laboratory of Computational Engineering, Helsinki University of Technology, Espoo, Finland.
Brain Res Cogn Brain Res. 2002 May;13(3):417-25. doi: 10.1016/s0926-6410(02)00053-8.
Seeing a talker's articulatory gestures may affect the observer's auditory speech percept. Observing congruent articulatory gestures may enhance the recognition of speech sounds [J. Acoust. Soc. Am. 26 (1954) 212], whereas observing incongruent gestures may change the auditory percept phonetically as occurs in the McGurk effect [Nature 264 (1976) 746]. For example, simultaneous acoustic /ba/ and visual /ga/ are usually heard as /da/. We studied cortical processing of occasional changes in audiovisual and visual speech stimuli with magnetoencephalography. In the audiovisual experiment congruent (acoustic /iti/, visual /iti/) and incongruent (acoustic /ipi/, visual /iti/) audiovisual stimuli, which were both perceived as /iti/, were presented among congruent /ipi/ (acoustic /ipi/, visual /ipi/) stimuli. In the visual experiment only the visual components of these stimuli were presented. A visual change both in audiovisual and visual experiments activated supratemporal auditory cortices bilaterally. The auditory cortex activation to a visual change occurred later in the visual than in the audiovisual experiment, suggesting that interaction between modalities accelerates the detection of visual change in speech.
看到说话者的发音手势可能会影响观察者的听觉言语感知。观察一致的发音手势可能会增强语音的识别[《美国声学学会杂志》26 (1954) 212]，而观察不一致的手势可能会像在麦格克效应中那样在语音上改变听觉感知[《自然》264 (1976) 746]。例如，同时呈现听觉/ba/和视觉/ga/通常会被听成/da/。我们用脑磁图研究了视听和视觉言语刺激中偶尔变化的皮层处理过程。在视听实验中，一致的（听觉/iti/，视觉/iti/）和不一致的（听觉/ipi/，视觉/iti/）视听刺激，二者都被感知为/iti/，被呈现在一致的/ipi/（听觉/ipi/，视觉/ipi/）刺激之中。在视觉实验中，只呈现这些刺激的视觉成分。视听实验和视觉实验中的视觉变化均双侧激活颞上听觉皮层。视觉变化引起的听觉皮层激活在视觉实验中比在视听实验中出现得更晚，这表明不同感觉模态之间的相互作用加速了言语中视觉变化的检测。
