Department of Management, The University of Tokyo, Tokyo, 981-0935, Japan.
Sensory Information Processing, John B. Pierce Laboratory, New Haven, CT 06519, USA.
Multisens Res. 2023 Nov 10;37(1):1-23. doi: 10.1163/22134808-bja10114.
The past two decades have seen an explosion of research on cross-modal correspondences. Broadly speaking, this term has been used to encompass associations between and among features, dimensions, or attributes across the senses. There has been an increasing interest in this topic amongst researchers from multiple fields (psychology, neuroscience, music, art, environmental design, etc.) and, importantly, an increasing breadth of the topic's scope. Here, this narrative review aims to reflect on what cross-modal correspondences are, where they come from, and what underlies them. We suggest that cross-modal correspondences are usefully conceived as relative associations between different actual or imagined sensory stimuli, many of these correspondences being shared by most people. A taxonomy of correspondences with four major kinds of associations (physiological, semantic, statistical, and affective) characterizes cross-modal correspondences. Sensory dimensions (quantity/quality) and sensory features (lower perceptual/higher cognitive) correspond in cross-modal correspondences. Cross-modal correspondences may be understood (or measured) from two complementary perspectives: the phenomenal view (perceptual experiences of subjective matching) and the behavioural response view (observable patterns of behavioural response to multiple sensory stimuli). Importantly, we reflect on remaining questions and standing issues that need to be addressed in order to develop an explanatory framework for cross-modal correspondences. Future research needs (a) to understand better when (and why) phenomenal and behavioural measures are coincidental and when they are not, and, ideally, (b) to determine whether different kinds of cross-modal correspondence (quantity/quality, lower perceptual/higher cognitive) rely on the same or different mechanisms.
过去二十年见证了跨模态对应关系研究的爆炸式增长。广义而言，这个术语被用于涵盖不同感觉之间的特征、维度或属性的关联。来自多个领域（心理学、神经科学、音乐、艺术、环境设计等）的研究人员对这个话题越来越感兴趣，而且，这个话题的范围也越来越广泛。在这里，本叙述性评论旨在反思跨模态对应关系是什么、它们来自何处以及它们的基础是什么。我们认为，跨模态对应关系可以被有益地理解为不同实际或想象的感觉刺激之间的相对关联，其中许多对应关系为大多数人所共有。对应关系的分类法具有四种主要类型的关联（生理、语义、统计和情感），其特征是跨模态对应关系。感觉维度（数量/质量）和感觉特征（较低的感知/较高的认知）在跨模态对应关系中对应。跨模态对应关系可以从两个互补的角度来理解（或测量）：现象学观点（主观匹配的感知体验）和行为反应观点（对多种感觉刺激的可观察行为反应模式）。重要的是，我们反思了需要解决的剩余问题和悬而未决的问题，以便为跨模态对应关系发展一个解释框架。未来的研究需要（a）更好地理解现象学和行为学测量何时（以及为何）巧合，何时不巧合，并且，理想情况下，（b）确定不同类型的跨模态对应关系（数量/质量、较低的感知/较高的认知）是否依赖于相同或不同的机制。
