Department of Psychology, Western University, London, Ontario, Canada; Brain and Mind Institute, Western University, London, Ontario, Canada.
Department of Psychology, Western University, London, Ontario, Canada; Neuroscience Program, Western University, London, Ontario, Canada.
Neuropsychologia. 2022 Sep 9;174:108336. doi: 10.1016/j.neuropsychologia.2022.108336. Epub 2022 Jul 22.
Integrating sensory information from multiple modalities leads to more precise and efficient perception and behaviour. The process of determining which sensory information should be perceptually bound is reliant on both low-level stimulus features, as well as multisensory associations learned throughout development based on the statistics of our environment. Here, we explored the relationship between multisensory associative learning and multisensory integration using encephalography (EEG) and behavioural measures. Sixty-one participants completed a three-phase study. First, participants were exposed to novel audiovisual shape-tone pairings with frequent and infrequent stimulus pairings and completed a target detection task. EEG recordings of the mismatch negativity (MMN) and P3 were calculated as neural indices of multisensory associative learning. Next, the same learned stimulus pairs were presented in audiovisual as well as unisensory auditory and visual modalities while both early (<100 ms) and late neural indices of multisensory integration were recorded. Finally, participants completed an analogous behavioural speeded-response task, with behavioural indices of multisensory gain calculated using the Race Model. Significant relationships were found in fronto-central and occipital areas between neural measures of associative learning and both early and late indices of multisensory integration in frontal and centro-parietal areas, respectively. Participants who showed stronger indices of associative learning also exhibited stronger indices of multisensory integration of the stimuli they learned to associate. Furthermore, a significant relationship was found between neural index of early multisensory integration and behavioural indices of multisensory gain. These results provide insight into the neural underpinnings of how higher-order processes such as associative learning guide multisensory integration.
多模态感官信息的整合导致更精确和高效的感知和行为。确定哪些感官信息应该被感知绑定的过程既依赖于低水平的刺激特征，也依赖于基于环境统计数据在整个发育过程中习得的多感官关联。在这里，我们使用脑电图（EEG）和行为测量来探索多感官联想学习和多感官整合之间的关系。61 名参与者完成了一个三阶段的研究。首先，参与者接触到具有频繁和不频繁刺激配对的新视听形状-音调配对，并完成了目标检测任务。计算了错配负波（MMN）和 P3 的 EEG 记录，作为多感官联想学习的神经指标。接下来，在视听以及单感官听觉和视觉模态中呈现相同的学习刺激对，同时记录早期（<100ms）和晚期的多感官整合的神经指标。最后，参与者完成了类似的行为加速反应任务，使用 Race 模型计算多感官增益的行为指标。在额中央和枕区发现了与关联学习的神经测量值与额中和顶-旁区域的早期和晚期多感官整合的指数之间存在显著关系。表现出更强关联学习指数的参与者也表现出他们学习关联的刺激的更强的多感官整合指数。此外，早期多感官整合的神经指数与多感官增益的行为指数之间存在显著关系。这些结果为多感官联想学习如何指导多感官整合等高级过程的神经基础提供了深入了解。
