Brain Language Laboratory, Department of Philosophy and Humanities, WE4 Freie Universität Berlin, Habelschwerdter Allee 45, 14195, Berlin, Germany.
Berlin School of Mind and Brain, Humboldt Universität zu Berlin, Luisenstraße 56, 10117, Berlin, Germany.
Sci Rep. 2019 Nov 8;9(1):16285. doi: 10.1038/s41598-019-52158-w.
During everyday social interaction, gestures are a fundamental part of human communication. The communicative pragmatic role of hand gestures and their interaction with spoken language has been documented at the earliest stage of language development, in which two types of indexical gestures are most prominent: the pointing gesture for directing attention to objects and the give-me gesture for making requests. Here we study, in adult human participants, the neurophysiological signatures of gestural-linguistic acts of communicating the pragmatic intentions of naming and requesting by simultaneously presenting written words and gestures. Already at ~150 ms, brain responses diverged between naming and request actions expressed by word-gesture combination, whereas the same gestures presented in isolation elicited their earliest neurophysiological dissociations significantly later (at ~210 ms). There was an early enhancement of request-evoked brain activity as compared with naming, which was due to sources in the frontocentral cortex, consistent with access to action knowledge in request understanding. In addition, an enhanced N400-like response indicated late semantic integration of gesture-language interaction. The present study demonstrates that word-gesture combinations used to express communicative pragmatic intentions speed up the brain correlates of comprehension processes - compared with gesture-only understanding - thereby calling into question current serial linguistic models viewing pragmatic function decoding at the end of a language comprehension cascade. Instead, information about the social-interactive role of communicative acts is processed instantaneously.
在日常社交互动中，手势是人类交流的基本组成部分。手势的交际语用作用及其与口语的相互作用在语言发展的最早阶段就已经被记录下来，其中两种类型的索引手势最为突出：指向手势用于将注意力引向物体，索取手势用于提出请求。在这里，我们研究了成年人类参与者在同时呈现书面单词和手势时，通过手势传达命名和请求的语用意图的手势语言行为的神经生理特征。在大约 150 毫秒时，单词-手势组合表达的命名和请求动作的大脑反应就已经出现了分歧，而单独呈现的相同手势则要晚得多（约 210 毫秒）才出现最早的神经生理分离。与命名相比，请求引起的大脑活动会提前增强，这归因于额中央皮质的源，与请求理解中对动作知识的访问一致。此外，增强的 N400 样反应表明手势语言相互作用的后期语义整合。本研究表明，用于表达交际语用意图的单词-手势组合与仅理解手势相比，加快了理解过程的大脑相关性，从而对当前将语用功能解码视为语言理解级联末端的串行语言模型提出了质疑。相反，关于交际行为的社会互动作用的信息是即时处理的。
