Brožová Natálie, Vollmer Lukas, Kampa Björn, Kayser Christoph, Fels Janina
Institute for Hearing Technology and Acoustics, RWTH Aachen University, Aachen, Germany.
Systems Neurophysiology Department, Institute of Zoology, RWTH Aachen University, Aachen, Germany.
Front Neurosci. 2025 Feb 20;19:1513083. doi: 10.3389/fnins.2025.1513083. eCollection 2025.
Audiovisual cross-modal correspondences (CMCs) refer to the brain's inherent ability to subconsciously connect auditory and visual information. These correspondences reveal essential aspects of multisensory perception and influence behavioral performance, enhancing reaction times and accuracy. However, the impact of different types of CMCs-arising from statistical co-occurrences or shaped by semantic associations-on information processing and decision-making remains underexplored. This study utilizes the Implicit Association Test, where unisensory stimuli are sequentially presented and linked via CMCs within an experimental block by the specific response instructions (either congruent or incongruent). Behavioral data are integrated with EEG measurements through neurally informed drift-diffusion modeling to examine how neural activity across both auditory and visual trials is modulated by CMCs. Our findings reveal distinct neural components that differentiate between congruent and incongruent stimuli regardless of modality, offering new insights into the role of congruency in shaping multisensory perceptual decision-making. Two key neural stages were identified: an Early component enhancing sensory encoding in congruent trials and a Late component affecting evidence accumulation, particularly in incongruent trials. These results suggest that cross-modal congruency primarily influences the processing and accumulation of sensory information rather than altering decision thresholds.
视听跨模态对应（CMCs）是指大脑潜意识地将听觉和视觉信息联系起来的内在能力。这些对应揭示了多感官感知的基本方面，并影响行为表现，提高反应速度和准确性。然而，不同类型的CMCs（由统计共现产生或由语义关联塑造）对信息处理和决策的影响仍未得到充分探索。本研究利用内隐联想测验，通过特定的反应指令（一致或不一致），在实验块内依次呈现单感官刺激并通过CMCs将它们联系起来。行为数据通过神经信息漂移扩散模型与脑电图测量相结合，以研究跨听觉和视觉试验的神经活动如何受到CMCs的调节。我们的研究结果揭示了不同的神经成分，这些成分能够区分一致和不一致的刺激，而不管其模态如何，这为一致性在塑造多感官感知决策中的作用提供了新的见解。确定了两个关键的神经阶段：一个早期成分在一致试验中增强感官编码，一个晚期成分影响证据积累，特别是在不一致试验中。这些结果表明，跨模态一致性主要影响感官信息的处理和积累，而不是改变决策阈值。
