Badde Stephanie, Navarro Karen T, Landy Michael S
Department of Psychology and Center of Neural Science, New York University, 6 Washington Place, New York, NY, 10003, USA.
Department of Psychology, University of Minnesota, 75 E River Rd., Minneapolis, MN, 55455, USA.
Cognition. 2020 Apr;197:104170. doi: 10.1016/j.cognition.2019.104170. Epub 2020 Feb 6.
At any moment in time, streams of information reach the brain through the different senses. Given this wealth of noisy information, it is essential that we select information of relevance - a function fulfilled by attention - and infer its causal structure to eventually take advantage of redundancies across the senses. Yet, the role of selective attention during causal inference in cross-modal perception is unknown. We tested experimentally whether the distribution of attention across vision and touch enhances cross-modal spatial integration (visual-tactile ventriloquism effect, Expt. 1) and recalibration (visual-tactile ventriloquism aftereffect, Expt. 2) compared to modality-specific attention, and then used causal-inference modeling to isolate the mechanisms behind the attentional modulation. In both experiments, we found stronger effects of vision on touch under distributed than under modality-specific attention. Model comparison confirmed that participants used Bayes-optimal causal inference to localize visual and tactile stimuli presented as part of a visual-tactile stimulus pair, whereas simultaneously collected unity judgments - indicating whether the visual-tactile pair was perceived as spatially-aligned - relied on a sub-optimal heuristic. The best-fitting model revealed that attention modulated sensory and cognitive components of causal inference. First, distributed attention led to an increase of sensory noise compared to selective attention toward one modality. Second, attending to both modalities strengthened the stimulus-independent expectation that the two signals belong together, the prior probability of a common source for vision and touch. Yet, only the increase in the expectation of vision and touch sharing a common source was able to explain the observed enhancement of visual-tactile integration and recalibration effects with distributed attention. In contrast, the change in sensory noise explained only a fraction of the observed enhancements, as its consequences vary with the overall level of noise and stimulus congruency. Increased sensory noise leads to enhanced integration effects for visual-tactile pairs with a large spatial discrepancy, but reduced integration effects for stimuli with a small or no cross-modal discrepancy. In sum, our study indicates a weak a priori association between visual and tactile spatial signals that can be strengthened by distributing attention across both modalities.
在任何时刻，信息流通过不同的感官传入大脑。鉴于存在如此丰富的嘈杂信息，我们必须选择相关信息——这是注意力所履行的功能——并推断其因果结构，以便最终利用各感官间的冗余信息。然而，选择性注意力在跨模态感知的因果推理过程中所起的作用尚不清楚。我们通过实验测试了与特定模态注意力相比，视觉和触觉之间注意力的分配是否会增强跨模态空间整合（视觉 - 触觉腹语效应，实验1）和重新校准（视觉 - 触觉腹语后效应，实验2），然后使用因果推理模型来分离注意力调制背后的机制。在两个实验中，我们发现与特定模态注意力相比，在分布式注意力下视觉对触觉的影响更强。模型比较证实，参与者使用贝叶斯最优因果推理来定位作为视觉 - 触觉刺激对一部分呈现的视觉和触觉刺激，而同时收集的一致性判断——表明视觉 - 触觉对是否被感知为空间对齐——则依赖于次优启发式方法。最佳拟合模型表明，注意力调制了因果推理的感官和认知成分。首先，与对一种模态的选择性注意力相比，分布式注意力导致感官噪声增加。其次，关注两种模态增强了刺激无关的期望，即两个信号属于同一来源，这是视觉和触觉共同来源的先验概率。然而，只有视觉和触觉共享共同来源的期望增加才能解释在分布式注意力下观察到的视觉 - 触觉整合和重新校准效应的增强。相比之下，感官噪声的变化仅解释了观察到的增强效应的一部分，因为其后果随噪声的总体水平和刺激一致性而变化。感官噪声增加会导致具有大空间差异的视觉 - 触觉对的整合效应增强，但对于具有小或无跨模态差异的刺激，整合效应会降低。总之，我们的研究表明视觉和触觉空间信号之间存在较弱的先验关联，通过在两种模态上分配注意力可以加强这种关联。
