University of Antwerp, Antwerp, Belgium; Peterhouse, University of Cambridge, Cambridge, UK.
Cortex. 2018 Aug;105:125-134. doi: 10.1016/j.cortex.2017.07.006. Epub 2017 Jul 17.
When I am looking at my coffee machine that makes funny noises, this is an instance of multisensory perception - I perceive this event by means of both vision and audition. But very often we only receive sensory stimulation from a multisensory event by means of one sense modality, for example, when I hear the noisy coffee machine in the next room, that is, without seeing it. The aim of this paper is to bring together empirical findings about multimodal perception and empirical findings about (visual, auditory, tactile) mental imagery and argue that on occasions like this, we have multimodal mental imagery: perceptual processing in one sense modality (here: vision) that is triggered by sensory stimulation in another sense modality (here: audition). Multimodal mental imagery is not a rare and obscure phenomenon. The vast majority of what we perceive are multisensory events: events that can be perceived in more than one sense modality - like the noisy coffee machine. And most of the time we are only acquainted with these multisensory events via a subset of the sense modalities involved - all the other aspects of these multisensory events are represented by means of multisensory mental imagery. This means that multisensory mental imagery is a crucial element of almost all instances of everyday perception.
当我看着那台发出奇怪声音的咖啡机时，这就是一种多感官感知的例子——我通过视觉和听觉来感知这个事件。但很多时候，我们只能通过一种感觉模态来接收多感官事件的感觉刺激，例如，当我听到隔壁房间里嘈杂的咖啡机声，但并没有看到它。本文的目的是将关于多模态感知的实证发现和关于（视觉、听觉、触觉）心理意象的实证发现结合起来，并认为在这种情况下，我们有了多模态心理意象：一种在一种感觉模态（这里是视觉）中的感知处理，是由另一种感觉模态（这里是听觉）的感觉刺激引发的。多模态心理意象并不是一种罕见和模糊的现象。我们感知到的绝大多数都是多感官事件：可以通过多种感觉模态感知的事件，就像嘈杂的咖啡机一样。而且大多数时候，我们只能通过所涉及的感觉模态的一个子集来了解这些多感官事件——这些多感官事件的其他方面都是通过多模态心理意象来表示的。这意味着多模态心理意象是几乎所有日常感知实例的一个关键要素。
