Basque Center on Cognition, Brain and Language, Paseo Mikeletegi 69, 2nd floor, 20009 Donostia, Spain; Tilburg University, Department of Cognitive Neuropsychology, P.O. Box 90153, Warandelaan 2, 5000 LE, Tilburg, the Netherlands.
Tilburg University, Department of Cognitive Neuropsychology, P.O. Box 90153, Warandelaan 2, 5000 LE, Tilburg, the Netherlands.
Neuropsychologia. 2014 Jan;53:115-21. doi: 10.1016/j.neuropsychologia.2013.11.011. Epub 2013 Nov 27.
Lip-read speech is integrated with heard speech at various neural levels. Here, we investigated the extent to which lip-read induced modulations of the auditory N1 and P2 (measured with EEG) are indicative of speech-specific audiovisual integration, and we explored to what extent the ERPs were modulated by phonetic audiovisual congruency. In order to disentangle speech-specific (phonetic) integration from non-speech integration, we used Sine-Wave Speech (SWS) that was perceived as speech by half of the participants (they were in speech-mode), while the other half was in non-speech mode. Results showed that the N1 obtained with audiovisual stimuli peaked earlier than the N1 evoked by auditory-only stimuli. This lip-read induced speeding up of the N1 occurred for listeners in speech and non-speech mode. In contrast, if listeners were in speech-mode, lip-read speech also modulated the auditory P2, but not if listeners were in non-speech mode, thus revealing speech-specific audiovisual binding. Comparing ERPs for phonetically congruent audiovisual stimuli with ERPs for incongruent stimuli revealed an effect of phonetic stimulus congruency that started at ~200 ms after (in)congruence became apparent. Critically, akin to the P2 suppression, congruency effects were only observed if listeners were in speech mode, and not if they were in non-speech mode. Using identical stimuli, we thus confirm that audiovisual binding involves (partially) different neural mechanisms for sound processing in speech and non-speech mode.
唇读语音与听到的语音在各种神经水平上整合在一起。在这里，我们研究了唇读引起的听觉 N1 和 P2 （通过 EEG 测量）调制在多大程度上表明语音特定的视听整合，以及我们探索了 ERPs 在多大程度上受到语音视听一致性的调制。为了将语音特定的（语音）整合与非语音整合分开，我们使用正弦波语音（Sine-Wave Speech，SWS），它被一半的参与者感知为语音（他们处于语音模式），而另一半处于非语音模式。结果表明，视听刺激引起的 N1 比仅听觉刺激引起的 N1 更早地达到峰值。这种唇读引起的 N1 加速发生在语音和非语音模式下的听众中。相比之下，如果听众处于语音模式，唇读语音也会调制听觉 P2，但如果听众处于非语音模式，则不会，从而揭示了语音特定的视听结合。将语音一致的视听刺激的 ERP 与语音不一致的刺激的 ERP 进行比较，揭示了语音刺激一致性的影响，该影响始于（语音）一致或不一致变得明显后约 200 毫秒。至关重要的是，与 P2 抑制类似，如果听众处于语音模式，则会观察到一致性效应，但如果他们处于非语音模式，则不会。因此，使用相同的刺激，我们确认视听结合涉及语音和非语音模式下声音处理的（部分）不同神经机制。
