Donders Institute, Radboud University, Nijmegen, The Netherlands.
Department of Cognition, Development and Educational Psychology, University of Barcelona, Barcelona, Spain.
Commun Biol. 2023 Jan 6;6(1):12. doi: 10.1038/s42003-022-04335-3.
Sounds enhance the detection of visual stimuli while concurrently biasing an observer's decisions. To investigate the neural mechanisms that underlie such multisensory interactions, we decoded time-resolved Signal Detection Theory sensitivity and criterion parameters from magneto-encephalographic recordings of participants that performed a visual detection task. We found that sounds improved visual detection sensitivity by enhancing the accumulation and maintenance of perceptual evidence over time. Meanwhile, criterion decoding analyses revealed that sounds induced brain activity patterns that resembled the patterns evoked by an actual visual stimulus. These two complementary mechanisms of audiovisual interplay differed in terms of their automaticity: Whereas the sound-induced enhancement in visual sensitivity depended on participants being actively engaged in a detection task, we found that sounds activated the visual cortex irrespective of task demands, potentially inducing visual illusory percepts. These results challenge the classical assumption that sound-induced increases in false alarms exclusively correspond to decision-level biases.
声音增强了对视觉刺激的检测，同时也影响了观察者的决策。为了研究这种多感官相互作用的神经机制，我们从执行视觉检测任务的参与者的脑磁图记录中解码了时间分辨的信号检测理论敏感性和标准参数。我们发现，声音通过随时间增强感知证据的积累和维持，提高了视觉检测的敏感性。同时，标准解码分析表明，声音诱导了类似于实际视觉刺激引起的脑活动模式。这两种视听相互作用的互补机制在自动性方面有所不同：虽然声音引起的视觉敏感性增强取决于参与者是否积极参与检测任务，但我们发现声音会激活视觉皮层，而不考虑任务需求，这可能会引起视觉错觉。这些结果挑战了经典的假设，即声音引起的虚报增加仅对应于决策水平的偏差。
