Bernardis Paolo, Gentilucci Maurizio
Dipartimento di Neuroscienze, Università di Parma, via Volturno 39, 43100 Parma, Italy.
Neuropsychologia. 2006;44(2):178-90. doi: 10.1016/j.neuropsychologia.2005.05.007. Epub 2005 Jul 7.
Humans speak and produce symbolic gestures. Do these two forms of communication interact, and how? First, we tested whether the two communication signals influenced each other when emitted simultaneously. Participants either pronounced words, or executed symbolic gestures, or emitted the two communication signals simultaneously. Relative to the unimodal conditions, multimodal voice spectra were enhanced by gestures, whereas multimodal gesture parameters were reduced by words. In other words, gesture reinforced word, whereas word inhibited gesture. In contrast, aimless arm movements and pseudo-words had no comparable effects. Next, we tested whether observing word pronunciation during gesture execution affected verbal responses in the same way as emitting the two signals. Participants responded verbally to either spoken words, or to gestures, or to the simultaneous presentation of the two signals. We observed the same reinforcement in the voice spectra as during simultaneous emission. These results suggest that spoken word and symbolic gesture are coded as single signal by a unique communication system. This signal represents the intention to engage a closer interaction with a hypothetical interlocutor and it may have a meaning different from when word and gesture are encoded singly.
人类会说话并做出象征性手势。这两种交流形式会相互作用吗？又是如何相互作用的呢？首先，我们测试了这两种交流信号在同时发出时是否会相互影响。参与者要么说出单词，要么做出象征性手势，要么同时发出这两种交流信号。相对于单模态条件，多模态语音频谱通过手势得到增强，而多模态手势参数则因单词而降低。换句话说，手势强化了单词，而单词抑制了手势。相比之下，无目的的手臂动作和伪词则没有类似的效果。接下来，我们测试了在执行手势时观察单词发音是否会与同时发出这两种信号一样影响言语反应。参与者对说出的单词、手势或这两种信号的同时呈现做出言语反应。我们观察到语音频谱中的强化现象与同时发出信号时相同。这些结果表明，口语单词和象征性手势被一个独特的交流系统编码为单一信号。这个信号代表了与一个假设的对话者进行更密切互动的意图，并且它可能具有与单词和手势单独编码时不同的含义。
