Brown David J, Simpson Andrew J R, Proulx Michael J
Crossmodal Cognition Lab, Department of Psychology, University of Bath Bath, UK ; Biological and Experimental Psychology Group, School of Biological and Chemical Sciences, Queen Mary University of London London, UK.
Centre for Vision, Speech and Signal Processing, University of Surrey Guildford, UK.
Front Psychol. 2015 Oct 13;6:1522. doi: 10.3389/fpsyg.2015.01522. eCollection 2015.
A critical task for the brain is the sensory representation and identification of perceptual objects in the world. When the visual sense is impaired, hearing and touch must take primary roles and in recent times compensatory techniques have been developed that employ the tactile or auditory system as a substitute for the visual system. Visual-to-auditory sonifications provide a complex, feature-based auditory representation that must be decoded and integrated into an object-based representation by the listener. However, we don't yet know what role the auditory system plays in the object integration stage and whether the principles of auditory scene analysis apply. Here we used coarse sonified images in a two-tone discrimination task to test whether auditory feature-based representations of visual objects would be confounded when their features conflicted with the principles of auditory consonance. We found that listeners (N = 36) performed worse in an object recognition task when the auditory feature-based representation was harmonically consonant. We also found that this conflict was not negated with the provision of congruent audio-visual information. The findings suggest that early auditory processes of harmonic grouping dominate the object formation process and that the complexity of the signal, and additional sensory information have limited effect on this.
大脑的一项关键任务是对世界中的感知对象进行感官表征和识别。当视觉受损时，听觉和触觉必须发挥主要作用，近年来已经开发出一些补偿技术，利用触觉或听觉系统来替代视觉系统。视觉到听觉的声化提供了一种基于特征的复杂听觉表征，听众必须对其进行解码并整合到基于对象的表征中。然而，我们尚不清楚听觉系统在对象整合阶段发挥什么作用，以及听觉场景分析的原理是否适用。在这里，我们在双音辨别任务中使用了粗略的声化图像，以测试当视觉对象的基于听觉特征的表征与听觉和谐原则冲突时，它们是否会被混淆。我们发现，当基于听觉特征的表征在和声上和谐时，听众（N = 36）在对象识别任务中的表现更差。我们还发现，提供一致的视听信息并不能消除这种冲突。研究结果表明，和声分组的早期听觉过程主导了对象形成过程，并且信号的复杂性和额外的感官信息对此影响有限。
