Plaisier Myrthe A, van Dam Loes C J, Glowania Catharina, Ernst Marc O
Faculty of Human Movement Sciences, Research Institute MOVE, VU University, Amsterdam, The Netherlands Cognitive Neuroscience Department and Cognitive Interaction Technology-Center of Excellence, Bielefeld University, Bielefeld, Germany.
Cognitive Neuroscience Department and Cognitive Interaction Technology-Center of Excellence, Bielefeld University, Bielefeld, Germany.
J Vis. 2014 Nov 20;14(13):22. doi: 10.1167/14.13.22.
We experience the world mostly in a multisensory fashion using a combination of all of our senses. Depending on the modality we can select different exploration strategies for extracting perceptual information. For instance, using touch we can enclose an object in our hand to explore parts of the object in parallel. Alternatively, we can trace the object with a single finger to explore its parts in a serial fashion. In this study we investigated whether the exploration mode (parallel vs. serial) affects the way sensory signals are combined. To this end, participants visually and haptically explored surfaces that varied in roll angle and indicated which side of the surface was perceived as higher. In Experiment 1, the exploration mode was the same for both modalities (i.e., both parallel or both serial). In Experiment 2, we introduced a difference in exploration mode between the two modalities (visual exploration was parallel while haptic exploration was serial or vice versa). The results showed that visual and haptic signals were combined in a statistically optimal fashion only when the exploration modes were the same. In case of an asymmetry in the exploration modes across modalities, integration was suboptimal. This indicates that spatial-temporal discrepancies in the acquisition of information in the two senses (i.e., haptic and visual) can lead to the breakdown of sensory integration.
我们主要通过综合运用所有感官，以多感官的方式体验世界。根据不同的感知方式，我们可以选择不同的探索策略来提取感知信息。例如，通过触摸，我们可以用手握住一个物体，并行探索物体的各个部分。或者，我们可以用一根手指沿着物体移动，以串行的方式探索其各个部分。在本研究中，我们调查了探索模式（并行与串行）是否会影响感官信号的组合方式。为此，参与者通过视觉和触觉探索了滚动角度不同的表面，并指出表面的哪一侧被感知为更高。在实验1中，两种感知方式的探索模式相同（即都是并行或都是串行）。在实验2中，我们在两种感知方式之间引入了探索模式的差异（视觉探索是并行的，而触觉探索是串行的，反之亦然）。结果表明，只有当探索模式相同时，视觉和触觉信号才会以统计上最优的方式进行组合。在跨感知方式的探索模式存在不对称的情况下，整合效果次优。这表明两种感官（即触觉和视觉）在获取信息时的时空差异会导致感官整合的失败。
