Gunter Thomas C, Weinbrenner J E Douglas
Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.
J Cogn Neurosci. 2017 Aug;29(8):1355-1367. doi: 10.1162/jocn_a_01125. Epub 2017 Mar 30.
When people talk, their speech is often accompanied by gestures. Although it is known that co-speech gestures can influence face-to-face communication, it is currently unclear to what extent they are actively used and under which premises they are prioritized to facilitate communication. We investigated these open questions in two experiments that varied how pointing gestures disambiguate the utterances of an interlocutor. Participants, whose event-related brain responses were measured, watched a video, where an actress was interviewed about, for instance, classical literature (e.g., Goethe and Shakespeare). While responding, the actress pointed systematically to the left side to refer to, for example, Goethe, or to the right to refer to Shakespeare. Her final statement was ambiguous and combined with a pointing gesture. The P600 pattern found in Experiment 1 revealed that, when pointing was unreliable, gestures were only monitored for their cue validity and not used for reference tracking related to the ambiguity. However, when pointing was a valid cue (Experiment 2), it was used for reference tracking, as indicated by a reduced N400 for pointing. In summary, these findings suggest that a general prioritization mechanism is in use that constantly monitors and evaluates the use of communicative cues against communicative priors on the basis of accumulated error information.
人们说话时，言语通常会伴随着手势。尽管已知伴随言语的手势会影响面对面交流，但目前尚不清楚它们在多大程度上被积极使用，以及在哪些前提下它们会被优先用于促进交流。我们在两个实验中研究了这些开放性问题，这两个实验改变了指示性手势如何消除对话者话语歧义的方式。测量了与事件相关脑反应的参与者观看了一段视频，视频中一名女演员接受了关于例如古典文学（如歌德和莎士比亚）的采访。在回答过程中，女演员有系统地指向左边以指代例如歌德，或者指向右边以指代莎士比亚。她的最后陈述具有歧义，并伴随着一个指示性手势。实验1中发现的P600模式表明，当指示不可靠时，手势仅根据其线索有效性进行监测，而不用于与歧义相关的指称追踪。然而，当指示是一个有效线索时（实验2），它被用于指称追踪，如指示时N400的降低所示。总之，这些发现表明存在一种通用的优先级机制，该机制会根据累积的错误信息，不断对照交流先验知识监测和评估交流线索的使用情况。
