Wu Jie, Li Qitian, Fu Qiufang, Rose Michael, Jing Liping
State Key Laboratory of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, 100101, China.
Department of Psychology, Chinese Academy of Sciences, Beijing, 100101, China.
Multisens Res. 2021 Aug 12;35(1):79-107. doi: 10.1163/22134808-bja10061.
Although it has been demonstrated that multisensory information can facilitate object recognition and object memory, it remains unclear whether such facilitation effect exists in category learning. To address this issue, comparable car images and sounds were first selected by a discrimination task in Experiment 1. Then, those selected images and sounds were utilized in a prototype category learning task in Experiments 2 and 3, in which participants were trained with auditory, visual, and audiovisual stimuli, and were tested with trained or untrained stimuli within the same categories presented alone or accompanied with a congruent or incongruent stimulus in the other modality. In Experiment 2, when low-distortion stimuli (more similar to the prototypes) were trained, there was higher accuracy for audiovisual trials than visual trials, but no significant difference between audiovisual and auditory trials. During testing, accuracy was significantly higher for congruent trials than unisensory or incongruent trials, and the congruency effect was larger for untrained high-distortion stimuli than trained low-distortion stimuli. In Experiment 3, when high-distortion stimuli (less similar to the prototypes) were trained, there was higher accuracy for audiovisual trials than visual or auditory trials, and the congruency effect was larger for trained high-distortion stimuli than untrained low-distortion stimuli during testing. These findings demonstrated that higher degree of stimuli distortion resulted in more robust multisensory effect, and the categorization of not only trained but also untrained stimuli in one modality could be influenced by an accompanying stimulus in the other modality.
尽管已有研究表明多感官信息能够促进物体识别和物体记忆，但在类别学习中这种促进作用是否存在仍不明确。为解决这一问题，实验1通过一项辨别任务首先挑选出了可比的汽车图像和声音。然后，在实验2和实验3的原型类别学习任务中使用了这些挑选出的图像和声音，其中参与者接受听觉、视觉和视听刺激的训练，并在单独呈现或伴有另一模态的一致或不一致刺激的同一类别中，用经过训练或未经训练的刺激进行测试。在实验2中，当训练低失真刺激（更类似于原型）时，视听试验的准确率高于视觉试验，但视听试验和听觉试验之间没有显著差异。在测试期间，一致试验的准确率显著高于单感官或不一致试验，并且对于未经训练的高失真刺激，一致效应大于经过训练的低失真刺激。在实验3中，当训练高失真刺激（与原型不太相似）时，视听试验的准确率高于视觉或听觉试验，并且在测试期间，对于经过训练的高失真刺激，一致效应大于未经训练的低失真刺激。这些发现表明，刺激失真程度越高，多感官效应越强，并且一种模态中经过训练和未经训练的刺激的分类都可能受到另一模态中伴随刺激的影响。
