Suppr超能文献

说话者的手势风格会影响语言理解:来自手势-言语整合的事件相关电位证据。

A speaker's gesture style can affect language comprehension: ERP evidence from gesture-speech integration.

作者信息

Obermeier Christian, Kelly Spencer D, Gunter Thomas C

机构信息

Max-Planck-Institute for Human Cognitive and Brain Sciences, Department of Neuropsychology, Leipzig, Germany and.

Colgate University, Department of Psychology, NY 13346, Hamilton, USA.

出版信息

Soc Cogn Affect Neurosci. 2015 Sep;10(9):1236-43. doi: 10.1093/scan/nsv011. Epub 2015 Feb 16.

Abstract

In face-to-face communication, speech is typically enriched by gestures. Clearly, not all people gesture in the same way, and the present study explores whether such individual differences in gesture style are taken into account during the perception of gestures that accompany speech. Participants were presented with one speaker that gestured in a straightforward way and another that also produced self-touch movements. Adding trials with such grooming movements makes the gesture information a much weaker cue compared with the gestures of the non-grooming speaker. The Electroencephalogram was recorded as participants watched videos of the individual speakers. Event-related potentials elicited by the speech signal revealed that adding grooming movements attenuated the impact of gesture for this particular speaker. Thus, these data suggest that there is sensitivity to the personal communication style of a speaker and that affects the extent to which gesture and speech are integrated during language comprehension.

摘要

在面对面交流中,言语通常会通过手势得到丰富。显然,并非所有人的手势方式都相同,本研究探讨了在感知伴随言语的手势时,是否会考虑到这种手势风格的个体差异。研究人员向参与者展示了一位以直接方式做手势的说话者和另一位也会进行自我触摸动作的说话者。与不做修饰动作的说话者的手势相比,增加带有此类修饰动作的试验会使手势信息成为一个弱得多的线索。当参与者观看各个说话者的视频时,记录他们的脑电图。由语音信号引发的事件相关电位显示,添加修饰动作会减弱手势对这位特定说话者的影响。因此,这些数据表明,对说话者的个人交流风格存在敏感性,并且这会影响在语言理解过程中手势与言语整合的程度。

相似文献

1
A speaker's gesture style can affect language comprehension: ERP evidence from gesture-speech integration.
Soc Cogn Affect Neurosci. 2015 Sep;10(9):1236-43. doi: 10.1093/scan/nsv011. Epub 2015 Feb 16.
2
Gesture style can affect the integration of gestures and speech: the evidence from Chinese ERP research.
Neuroreport. 2020 Aug 12;31(12):885-890. doi: 10.1097/WNR.0000000000001458.
3
The role of iconic gestures in speech disambiguation: ERP evidence.
J Cogn Neurosci. 2007 Jul;19(7):1175-92. doi: 10.1162/jocn.2007.19.7.1175.
4
The benefit of gestures during communication: evidence from hearing and hearing-impaired individuals.
Cortex. 2012 Jul;48(7):857-70. doi: 10.1016/j.cortex.2011.02.007. Epub 2011 Feb 13.
5
Does dynamic information about the speaker's face contribute to semantic speech processing? ERP evidence.
Cortex. 2018 Jul;104:12-25. doi: 10.1016/j.cortex.2018.03.031. Epub 2018 Apr 9.
6
Neural correlates of bimodal speech and gesture comprehension.
Brain Lang. 2004 Apr;89(1):253-60. doi: 10.1016/S0093-934X(03)00335-3.
7
The role of synchrony and ambiguity in speech-gesture integration during comprehension.
J Cogn Neurosci. 2011 Aug;23(8):1845-54. doi: 10.1162/jocn.2010.21462. Epub 2010 Mar 4.
8
The communicative style of a speaker can affect language comprehension? ERP evidence from the comprehension of irony.
Brain Res. 2010 Jan 22;1311:121-35. doi: 10.1016/j.brainres.2009.10.077. Epub 2009 Nov 10.

引用本文的文献

1
Gesture links language and cognition for spoken and signed languages.
Nat Rev Psychol. 2023 Jul;2(7):407-420. doi: 10.1038/s44159-023-00186-9. Epub 2023 May 25.
2
The role of multimodal cues in second language comprehension.
Sci Rep. 2023 Nov 27;13(1):20824. doi: 10.1038/s41598-023-47643-2.
3
Zero-shot style transfer for gesture animation driven by text and speech using adversarial disentanglement of multimodal style encoding.
Front Artif Intell. 2023 Jun 12;6:1142997. doi: 10.3389/frai.2023.1142997. eCollection 2023.
4
Electrophysiological evidence for the enhancement of gesture-speech integration by linguistic predictability during multimodal discourse comprehension.
Cogn Affect Behav Neurosci. 2023 Apr;23(2):340-353. doi: 10.3758/s13415-023-01074-8. Epub 2023 Feb 23.
5
The role of hand gestures in emotion communication: Do type and size of gestures matter?
Psychol Res. 2023 Sep;87(6):1880-1898. doi: 10.1007/s00426-022-01774-9. Epub 2022 Nov 27.
6
More than words: word predictability, prosody, gesture and mouth movements in natural language comprehension.
Proc Biol Sci. 2021 Jul 28;288(1955):20210500. doi: 10.1098/rspb.2021.0500. Epub 2021 Jul 21.
7
Beat Gestures for Comprehension and Recall: Differential Effects of Language Learners and Native Listeners.
Front Psychol. 2020 Oct 19;11:575929. doi: 10.3389/fpsyg.2020.575929. eCollection 2020.
8
Gesture's body orientation modulates the N400 for visual sentences primed by gestures.
Hum Brain Mapp. 2020 Dec;41(17):4901-4911. doi: 10.1002/hbm.25166. Epub 2020 Aug 18.
9
Don't Get Me Wrong: ERP Evidence from Cueing Communicative Intentions.
Front Psychol. 2017 Sep 11;8:1465. doi: 10.3389/fpsyg.2017.01465. eCollection 2017.

本文引用的文献

1
Multisensory integration: the case of a time window of gesture-speech integration.
J Cogn Neurosci. 2015 Feb;27(2):292-307. doi: 10.1162/jocn_a_00688.
2
Verbal working memory predicts co-speech gesture: evidence from individual differences.
Cognition. 2014 Aug;132(2):174-80. doi: 10.1016/j.cognition.2014.03.012. Epub 2014 May 8.
3
Toward a more embedded/extended perspective on the cognitive function of gestures.
Front Psychol. 2014 Apr 24;5:359. doi: 10.3389/fpsyg.2014.00359. eCollection 2014.
4
Eye'm talking to you: speakers' gaze direction modulates co-speech gesture processing in the right MTG.
Soc Cogn Affect Neurosci. 2015 Feb;10(2):255-61. doi: 10.1093/scan/nsu047. Epub 2014 Mar 19.
5
The processing nature of the n400: evidence from masked priming.
J Cogn Neurosci. 1993 Winter;5(1):34-44. doi: 10.1162/jocn.1993.5.1.34.
6
7
Individual differences in the gesture effect on working memory.
Psychon Bull Rev. 2013 Jun;20(3):496-500. doi: 10.3758/s13423-012-0365-0.
8
When one person's mistake is another's standard usage: the effect of foreign accent on syntactic processing.
J Cogn Neurosci. 2012 Apr;24(4):878-87. doi: 10.1162/jocn_a_00103. Epub 2011 Aug 3.
9
The benefit of gestures during communication: evidence from hearing and hearing-impaired individuals.
Cortex. 2012 Jul;48(7):857-70. doi: 10.1016/j.cortex.2011.02.007. Epub 2011 Feb 13.
10
When do gestures communicate? A meta-analysis.
Psychol Bull. 2011 Mar;137(2):297-315. doi: 10.1037/a0022128.

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验