Swammerdam Institute for Life Sciences, Center for Neuroscience, Faculty of Science, University of Amsterdam, 1098 XH Amsterdam, The Netherlands.
Department of Psychology, Brain and Cognition, University of Amsterdam, 1098 XH Amsterdam, The Netherlands.
Multisens Res. 2022 Dec 1;36(1):31-56. doi: 10.1163/22134808-bja10088.
Perceptual decision-making in a dynamic environment requires two integration processes: integration of sensory evidence from multiple modalities to form a coherent representation of the environment, and integration of evidence across time to accurately make a decision. Only recently studies started to unravel how evidence from two modalities is accumulated across time to form a perceptual decision. One important question is whether information from individual senses contributes equally to multisensory decisions. We designed a new psychophysical task that measures how visual and auditory evidence is weighted across time. Participants were asked to discriminate between two visual gratings, and/or two sounds presented to the right and left ear based on respectively contrast and loudness. We varied the evidence, i.e., the contrast of the gratings and amplitude of the sound, over time. Results showed a significant increase in performance accuracy on multisensory trials compared to unisensory trials, indicating that discriminating between two sources is improved when multisensory information is available. Furthermore, we found that early evidence contributed most to sensory decisions. Weighting of unisensory information during audiovisual decision-making dynamically changed over time. A first epoch was characterized by both visual and auditory weighting, during the second epoch vision dominated and the third epoch finalized the weighting profile with auditory dominance. Our results suggest that during our task multisensory improvement is generated by a mechanism that requires cross-modal interactions but also dynamically evokes dominance switching.
将来自多个感觉模态的感觉证据整合起来，以形成对环境的连贯表示；以及跨时间整合证据，以准确做出决策。直到最近，研究才开始揭示来自两种感觉模态的证据如何随时间积累，从而形成感知决策。一个重要的问题是，个体感觉的信息是否平等地贡献于多感觉决策。我们设计了一种新的心理物理学任务，用于测量视觉和听觉证据如何随时间加权。参与者被要求根据对比度和响度分别辨别出呈现给右耳和左耳的两个视觉光栅和两个声音。我们随时间改变证据，即光栅的对比度和声音的幅度。结果表明，与单感觉试验相比，多感觉试验的性能准确性显著提高，表明当多感觉信息可用时，辨别两个来源的能力得到提高。此外，我们发现早期证据对感觉决策的贡献最大。在视听决策过程中，单感觉信息的加权随时间动态变化。第一阶段的特点是视觉和听觉加权，第二阶段是视觉主导，第三阶段则是以听觉为主导确定加权分布。我们的结果表明，在我们的任务中，多感觉的改善是由一种需要跨模态相互作用的机制产生的，但也会动态地引发主导权切换。
