Cognitive Neuroscience Laboratory, Graduate School of Interdisciplinary Science and Engineering in Health Systems, Okayama University, Okayama, Japan.
Department of Psychology, Research Center for Psychology and Behavioral Sciences, Soochow University, Suzhou, People's Republic of China.
Atten Percept Psychophys. 2022 Jul;84(5):1625-1634. doi: 10.3758/s13414-021-02437-4. Epub 2022 May 31.
Evidence has shown that multisensory integration benefits to unisensory perception performance are asymmetric and that auditory perception performance can receive more multisensory benefits, especially when the attention focus is directed toward a task-irrelevant visual stimulus. At present, whether the benefits of semantically (in)congruent multisensory integration with modal-based attention for subsequent unisensory short-term memory (STM) retrieval are also asymmetric remains unclear. Using a delayed matching-to-sample paradigm, the present study investigated this issue by manipulating the attention focus during multisensory memory encoding. The results revealed that both visual and auditory STM retrieval reaction times were faster under semantically congruent multisensory conditions than under unisensory memory encoding conditions. We suggest that coherent multisensory representation formation might be optimized by restricted multisensory encoding and can be rapidly triggered by subsequent unisensory memory retrieval demands. Crucially, auditory STM retrieval is exclusively accelerated by semantically congruent multisensory memory encoding, indicating that the less effective sensory modality of memory retrieval relies more on the coherent prior formation of a multisensory representation optimized by modal-based attention.
证据表明，多感觉整合对单感觉感知表现的益处是不对称的，而听觉感知表现可以获得更多的多感觉益处，特别是当注意力集中在与任务无关的视觉刺激上时。目前，模态注意下语义（不）一致的多感觉整合对后续单感觉短期记忆（STM）检索的益处是否也是不对称的还不清楚。本研究采用延迟匹配样本范式，通过在多感觉记忆编码期间操纵注意力焦点来研究这个问题。结果表明，在语义一致的多感觉条件下，视觉和听觉 STM 检索反应时均快于单感觉记忆编码条件。我们认为，受限制的多感觉编码可能会优化连贯的多感觉表现形成，并且可以快速触发后续单感觉记忆检索需求。至关重要的是，听觉 STM 检索仅通过语义一致的多感觉记忆编码得到加速，这表明记忆检索的效率较低的感官模态更依赖于模态注意优化的连贯的多感觉表现形成。
