Willems Roel M, Oostenveld Robert, Hagoort Peter
F. C. Donders Centre for Cognitive Neuroimaging, Radboud University Nijmegen, P.O. Box 9101, 6500 HB Nijmegen, The Netherlands.
Brain Res. 2008 Jul 11;1219:78-90. doi: 10.1016/j.brainres.2008.04.065. Epub 2008 May 1.
Language is often perceived together with visual information. This raises the question on how the brain integrates information conveyed in visual and/or linguistic format during spoken language comprehension. In this study we investigated the dynamics of semantic integration of visual and linguistic information by means of time-frequency analysis of the EEG signal. A modified version of the N400 paradigm with either a word or a picture of an object being semantically incongruous with respect to the preceding sentence context was employed. Event-Related Potential (ERP) analysis showed qualitatively similar N400 effects for integration of either word or picture. Time-frequency analysis revealed early specific decreases in alpha and gamma band power for linguistic and visual information respectively. We argue that these reflect a rapid context-based analysis of acoustic (word) or visual (picture) form information. We conclude that although full semantic integration of linguistic and visual information occurs through a common mechanism, early differences in oscillations in specific frequency bands reflect the format of the incoming information and, importantly, an early context-based detection of its congruity with respect to the preceding language context.
语言常常与视觉信息一同被感知。这就引出了一个问题：在口语理解过程中，大脑是如何整合以视觉和/或语言形式传达的信息的。在本研究中，我们通过对脑电图（EEG）信号进行时频分析，研究了视觉和语言信息语义整合的动态过程。我们采用了N400范式的一个修改版本，其中单词或物体图片在语义上与前文句子语境不一致。事件相关电位（ERP）分析表明，单词或图片整合时的N400效应在性质上相似。时频分析分别揭示了语言和视觉信息在阿尔法和伽马波段功率上的早期特定下降。我们认为，这些反映了基于上下文对声学（单词）或视觉（图片）形式信息的快速分析。我们得出结论，尽管语言和视觉信息的完全语义整合是通过一种共同机制发生的，但特定频段振荡的早期差异反映了传入信息的形式，重要的是，反映了基于上下文对其与前文语言语境一致性的早期检测。
