Department of Psychology, Leibniz Universität Hannover, Hannover, Germany.
Institute of Sports Science, Leibniz Universität Hannover, Hannover, Germany.
Psychol Res. 2022 Sep;86(6):1930-1943. doi: 10.1007/s00426-021-01618-y. Epub 2021 Dec 2.
Eye contact is a dynamic social signal that captures attention and plays a critical role in human communication. In particular, direct gaze often accompanies communicative acts in an ostensive function: a speaker directs her gaze towards the addressee to highlight the fact that this message is being intentionally communicated to her. The addressee, in turn, integrates the speaker's auditory and visual speech signals (i.e., her vocal sounds and lip movements) into a unitary percept. It is an open question whether the speaker's gaze affects how the addressee integrates the speaker's multisensory speech signals. We investigated this question using the classic McGurk illusion, an illusory percept created by presenting mismatching auditory (vocal sounds) and visual information (speaker's lip movements). Specifically, we manipulated whether the speaker (a) moved his eyelids up/down (i.e., open/closed his eyes) prior to speaking or did not show any eye motion, and (b) spoke with open or closed eyes. When the speaker's eyes moved (i.e., opened or closed) before an utterance, and when the speaker spoke with closed eyes, the McGurk illusion was weakened (i.e., addressees reported significantly fewer illusory percepts). In line with previous research, this suggests that motion (opening or closing), as well as the closed state of the speaker's eyes, captured addressees' attention, thereby reducing the influence of the speaker's lip movements on the addressees' audiovisual integration process. Our findings reaffirm the power of speaker gaze to guide attention, showing that its dynamics can modulate low-level processes such as the integration of multisensory speech signals.
眼神交流是一种动态的社交信号，它能吸引注意力，并在人类交流中起着至关重要的作用。特别是，直接注视通常伴随着交际行为的明示功能：说话者将目光投向听话者，以突出这一信息正被有意传达给她。反过来，听话者将说话者的听觉和视觉言语信号（即她的声音和唇动）整合为一个单一的感知。一个悬而未决的问题是，说话者的注视是否会影响听话者如何整合说话者的多感官言语信号。我们使用经典的 McGurk 错觉来研究这个问题，这是一种由不匹配的听觉（声音）和视觉信息（说话者的唇动）产生的错觉感知。具体来说，我们操纵了说话者（a）在说话前上下移动眼皮（即睁开/闭上眼睛）或不显示任何眼部运动，以及（b）睁眼或闭眼说话。当说话者的眼睛在说话前移动（即睁开或闭上），并且说话者闭着眼睛说话时，McGurk 错觉会减弱（即，被试报告的错觉感知明显减少）。与之前的研究一致，这表明运动（打开或关闭）以及说话者眼睛的闭合状态吸引了被试的注意力，从而减少了说话者唇动对被试视听整合过程的影响。我们的发现再次证实了说话者注视的引导注意力的力量，表明其动态可以调节低水平的过程，如多感官言语信号的整合。
