Gonzales Mariel G, Backer Kristina C, Yan Yueqi, Miller Lee M, Bortfeld Heather, Shahin Antoine J
Department of Cognitive and Information Sciences, University of California, 5200 N Lake Rd, Merced, CA 95343, USA.
Health Sciences Research Institute, University of California, 5200 N Lake Rd, Merced, CA 95343, USA.
iScience. 2022 Jun 26;25(7):104671. doi: 10.1016/j.isci.2022.104671. eCollection 2022 Jul 15.
Previous work addressing the influence of audition on visual perception has mainly been assessed using non-speech stimuli. Herein, we introduce the Audiovisual Time-Flow Illusion in spoken language, underscoring the role of audition in multisensory processing. When brief pauses were inserted into or brief portions were removed from an acoustic speech stream, individuals perceived the corresponding visual speech as or "", respectively-even though the visual stimulus was intact. When the stimulus manipulation was reversed-brief pauses were inserted into, or brief portions were removed from the visual speech stream-individuals failed to perceive the illusion in the corresponding intact auditory stream. Our findings demonstrate that in the context of spoken language, people continually realign the pace of their visual perception based on that of the auditory input. In short, the auditory modality sets the pace of the visual modality during audiovisual speech processing.
以往关于听觉对视觉感知影响的研究主要使用非语音刺激进行评估。在此，我们引入了口语中的视听时间流错觉，强调了听觉在多感官处理中的作用。当在语音流中插入短暂停顿或去除短暂片段时，即使视觉刺激完好无损，个体也会分别将相应的视觉语音感知为“”或“”。当刺激操作反过来——在视觉语音流中插入短暂停顿或去除短暂片段——个体无法在相应完好的听觉流中感知到这种错觉。我们的研究结果表明，在口语环境中，人们会根据听觉输入不断调整视觉感知的节奏。简而言之，在视听语音处理过程中，听觉模态设定了视觉模态的节奏。
