Gao Yi, Xue Kai, Odegaard Brian, Rahnev Dobromir
bioRxiv. 2025 Jan 12:2023.06.07.544029. doi: 10.1101/2023.06.07.544029.
It is well known that sensory information from one modality can automatically affect judgments from a different sensory modality. However, it remains unclear what determines the strength of the influence of an irrelevant sensory cue from one modality on a perceptual judgment for a different modality. Here we test whether the strength of multisensory impact by an irrelevant sensory cue depends on participants' objective accuracy or subjective confidence for that cue. We created visual motion stimuli with low vs. high overall motion energy, where high-energy stimuli yielded higher confidence but lower accuracy in a visual-only task. We then tested the impact of the low- and high-energy visual stimuli on auditory motion perception. We found that the high-energy visual stimuli influenced the auditory motion judgments more strongly than the low-energy visual stimuli, consistent with their higher confidence but contrary to their lower accuracy. A computational model assuming common principles underlying confidence reports and multisensory integration captured these effects. Our findings show that automatic multisensory integration follows subjective confidence rather than objective performance and suggest the existence of common computations across vastly different stages of perceptual decision making.
众所周知，来自一种感觉模态的感觉信息能够自动影响来自不同感觉模态的判断。然而，目前尚不清楚是什么决定了来自一种模态的无关感觉线索对不同模态的知觉判断的影响强度。在这里，我们测试了无关感觉线索的多感觉影响强度是否取决于参与者对该线索的客观准确性或主观信心。我们创建了具有低整体运动能量与高整体运动能量的视觉运动刺激，其中高能量刺激在仅视觉任务中产生更高的信心但更低的准确性。然后，我们测试了低能量和高能量视觉刺激对听觉运动知觉的影响。我们发现，高能量视觉刺激比低能量视觉刺激对听觉运动判断的影响更强，这与其更高的信心一致，但与其更低的准确性相反。一个假设信心报告和多感觉整合背后存在共同原则的计算模型捕捉到了这些效应。我们的研究结果表明，自动多感觉整合遵循主观信心而非客观表现，并表明在知觉决策的截然不同的阶段存在共同的计算过程。
