Kandana Arachchige Kendra Gimhani, Blekic Wivine, Simoes Loureiro Isabelle, Lefebvre Laurent
Department of Cognitive Psychology and Neuropsychology, University of Mons, Mons, Belgium.
Front Psychol. 2021 Nov 30;12:776867. doi: 10.3389/fpsyg.2021.776867. eCollection 2021.
Numerous studies have explored the benefit of iconic gestures in speech comprehension. However, only few studies have investigated how visual attention was allocated to these gestures in the context of clear versus degraded speech and the way information is extracted for enhancing comprehension. This study aimed to explore the effect of iconic gestures on comprehension and whether fixating the gesture is required for information extraction. Four types of gestures (i.e., semantically and syntactically incongruent iconic gestures, meaningless configurations, and congruent iconic gestures) were presented in a sentence context in three different listening conditions (i.e., clear, partly degraded or fully degraded speech). Using eye tracking technology, participants' gaze was recorded, while they watched video clips after which they were invited to answer simple comprehension questions. Results first showed that different types of gestures differently attract attention and that the more speech was degraded, the less participants would pay attention to gestures. Furthermore, semantically incongruent gestures appeared to particularly impair comprehension although not being fixated while congruent gestures appeared to improve comprehension despite also not being fixated. These results suggest that covert attention is sufficient to convey information that will be processed by the listener.
众多研究探讨了标志性手势在言语理解中的益处。然而,只有少数研究调查了在清晰语音与 degraded 语音的背景下,视觉注意力是如何分配到这些手势上的,以及信息是如何被提取以增强理解的。本研究旨在探讨标志性手势对理解的影响,以及信息提取是否需要注视手势。在三种不同的听力条件(即清晰、部分 degraded 或完全 degraded 语音)下,在句子语境中呈现了四种类型的手势(即语义和句法不一致的标志性手势、无意义的构型和一致的标志性手势)。使用眼动追踪技术,记录参与者观看视频片段时的注视情况,之后邀请他们回答简单的理解问题。结果首先表明,不同类型的手势吸引注意力的方式不同,语音 degraded 程度越高,参与者对手势的关注就越少。此外,语义不一致的手势似乎特别损害理解,尽管没有被注视,而一致的手势似乎能提高理解,尽管同样也没有被注视。这些结果表明,隐蔽注意力足以传达将被听众处理的信息。