Gao Mingjie, Zhu Weina, Drewes Jan
School of Information Science, Yunnan University, Kunming, China.
Institute of Brain and Psychological Sciences, Sichuan Normal University, Chengdu, China.
Heliyon. 2024 Jun 27;10(13):e33828. doi: 10.1016/j.heliyon.2024.e33828. eCollection 2024 Jul 15.
We compared the time course of cross-modal semantic effects induced by both naturalistic sounds and spoken words on the processing of visual stimuli, whether visible or suppressed form awareness through continuous flash suppression. We found that, under visible conditions, spoken words elicited audio-visual semantic effects over longer time (-1000, -500, -250 ms SOAs) than naturalistic sounds (-500, -250 ms SOAs). Performance was generally better with auditory primes, but more so with congruent stimuli. Spoken words presented in advance (-1000, -500 ms) outperformed naturalistic sounds; the opposite was true for (near-)simultaneous presentations. Congruent spoken words demonstrated superior categorization performance compared to congruent naturalistic sounds. The audio-visual semantic congruency effect still occurred with suppressed visual stimuli, although without significant variations in the temporal patterns between auditory types. These findings indicate that: 1. Semantically congruent auditory input can enhance visual processing performance, even when the visual stimulus is imperceptible to conscious awareness. 2. The temporal dynamics is contingent on the auditory types only when the visual stimulus is visible. 3. Audiovisual semantic integration requires sufficient time for processing auditory information.
我们比较了自然声音和口语单词对视觉刺激处理所引发的跨模态语义效应的时间进程，视觉刺激无论是可见的还是通过持续闪光抑制处于意识抑制状态。我们发现，在可见条件下，口语单词引发视听语义效应的时间（刺激开始异步时间为-1000、-500、-250毫秒）比自然声音（-500、-250毫秒）更长。一般来说，听觉启动词的表现更好，但一致性刺激的表现更佳。提前呈现（-1000、-500毫秒）的口语单词比自然声音表现更好；（近）同时呈现时则相反。与一致性自然声音相比，一致性口语单词表现出更优的分类性能。尽管听觉类型之间的时间模式没有显著差异，但在视觉刺激被抑制的情况下，视听语义一致性效应仍然会出现。这些发现表明：1. 语义一致的听觉输入可以提高视觉处理性能，即使视觉刺激在意识层面无法察觉。2. 只有当视觉刺激可见时，时间动态才取决于听觉类型。3. 视听语义整合需要足够的时间来处理听觉信息。
