Wu Ying Choon, Coulson Seana
Cognitive Science Department 0515, 9500 Gilman Drive, La Jolla, CA 92093-0515, USA.
Brain Lang. 2007 Jun;101(3):234-45. doi: 10.1016/j.bandl.2006.12.003. Epub 2007 Jan 12.
EEG was recorded as adults watched short segments of spontaneous discourse in which the speaker's gestures and utterances contained complementary information. Videos were followed by one of four types of picture probes: cross-modal related probes were congruent with both speech and gestures; speech-only related probes were congruent with information in the speech, but not the gesture; and two sorts of unrelated probes were created by pairing each related probe with a different discourse prime. Event-related potentials (ERPs) elicited by picture probes were measured within the time windows of the N300 (250-350 ms post-stimulus) and N400 (350-550 ms post-stimulus). Cross-modal related probes elicited smaller N300 and N400 than speech-only related ones, indicating that pictures were easier to interpret when they corresponded with gestures. N300 and N400 effects were not due to differences in the visual complexity of each probe type, since the same cross-modal and speech-only picture probes elicited N300 and N400 with similar amplitudes when they appeared as unrelated items. These findings extend previous research on gesture comprehension by revealing how iconic co-speech gestures modulate conceptualization, enabling listeners to better represent visuo-spatial aspects of the speaker's meaning.
在成年人观看自发话语的短片时记录脑电图，其中说话者的手势和话语包含互补信息。视频之后是四种类型的图片探测之一：跨模态相关探测与言语和手势都一致；仅言语相关探测与言语中的信息一致，但与手势不一致；两种不相关探测是通过将每个相关探测与不同的话语启动项配对而创建的。在N300（刺激后250 - 350毫秒）和N400（刺激后350 - 550毫秒）的时间窗口内测量图片探测引发的事件相关电位（ERP）。跨模态相关探测引发的N300和N400比仅言语相关探测引发的小，这表明当图片与手势对应时更容易解释。N300和N400效应不是由于每种探测类型的视觉复杂性差异，因为相同的跨模态和仅言语图片探测作为不相关项目出现时引发的N300和N400具有相似的幅度。这些发现通过揭示标志性的伴随言语手势如何调节概念化，扩展了先前关于手势理解的研究，使听众能够更好地表征说话者意思的视觉空间方面。
