Department of Psychology, University of California, San Diego.
Department of Psychological and Brain Sciences, Dartmouth College.
Psychol Sci. 2024 Jul;35(7):814-824. doi: 10.1177/09567976241237737. Epub 2024 Jun 18.
Despite the intuitive feeling that our visual experience is coherent and comprehensive, the world is full of ambiguous and indeterminate information. Here we explore how the visual system might take advantage of ambient sounds to resolve this ambiguity. Young adults (s = 20-30) were tasked with identifying an object slowly fading in through visual noise while a task-irrelevant sound played. We found that participants demanded more visual information when the auditory object was incongruent with the visual object compared to when it was not. Auditory scenes, which are only probabilistically related to specific objects, produced similar facilitation even for unheard objects (e.g., a bench). Notably, these effects traverse categorical and specific auditory and visual-processing domains as participants performed across-category and within-category visual tasks, underscoring cross-modal integration across multiple levels of perceptual processing. To summarize, our study reveals the importance of audiovisual interactions to support meaningful perceptual experiences in naturalistic settings.
尽管我们直观地感觉自己的视觉体验是连贯和全面的，但世界充满了模糊和不确定的信息。在这里，我们探讨了视觉系统如何利用环境声音来解决这种模糊性。我们让年轻成年人（s=20-30 岁）在观看视觉噪声中逐渐出现的物体的同时，听与任务无关的声音。我们发现，与听觉物体与视觉物体一致时相比，当听觉物体不一致时，参与者需要更多的视觉信息。即使对于未听到的物体（例如长凳），仅与特定物体存在概率关系的听觉场景也会产生类似的促进作用。值得注意的是，这些效果跨越了类别和特定的听觉和视觉处理领域，因为参与者在跨类别和类别内的视觉任务中表现，强调了跨多个感知处理水平的跨模态整合。总之，我们的研究揭示了视听相互作用对于在自然环境中支持有意义的感知体验的重要性。
