Centre for Computational Neuroscience and Cognitive Robotics, University of Birmingham, Birmingham, UK.
Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands.
Sci Rep. 2021 May 25;11(1):10832. doi: 10.1038/s41598-021-90183-w.
Information integration is considered a hallmark of human consciousness. Recent research has challenged this tenet by showing multisensory interactions in the absence of awareness. This psychophysics study assessed the impact of spatial and semantic correspondences on audiovisual binding in the presence and absence of visual awareness by combining forward-backward masking with spatial ventriloquism. Observers were presented with object pictures and synchronous sounds that were spatially and/or semantically congruent or incongruent. On each trial observers located the sound, identified the picture and rated the picture's visibility. We observed a robust ventriloquist effect for subjectively visible and invisible pictures indicating that pictures that evade our perceptual awareness influence where we perceive sounds. Critically, semantic congruency enhanced these visual biases on perceived sound location only when the picture entered observers' awareness. Our results demonstrate that crossmodal influences operating from vision to audition and vice versa are interactively controlled by spatial and semantic congruency in the presence of awareness. However, when visual processing is disrupted by masking procedures audiovisual interactions no longer depend on semantic correspondences.
信息整合被认为是人类意识的标志。最近的研究通过显示在没有意识的情况下存在多感觉相互作用，对这一观点提出了挑战。本项心理物理学研究通过结合前向-后向掩蔽和空间错觉，评估了在存在和不存在视觉意识的情况下，空间和语义对应对视听绑定的影响。观察者被呈现物体图片和同步声音，这些声音在空间上和/或语义上是一致的或不一致的。在每次试验中，观察者定位声音，识别图片并对图片的可见度进行评分。我们观察到一个强大的声音错觉效应，对于主观上可见和不可见的图片都表明，那些逃避我们感知的图片会影响我们感知声音的位置。关键的是，只有当图片进入观察者的意识时，语义一致性才会增强对感知声音位置的这些视觉偏差。我们的结果表明，在存在意识的情况下，来自视觉到听觉以及反之亦然的跨模态影响受到空间和语义一致性的交互控制。然而，当视觉处理被掩蔽程序破坏时，视听相互作用不再依赖于语义对应。
