Department of Preclinical Imaging and Radiopharmacy, University of Tübingen, Tübingen 72076, Germany.
Cognition and Consciousness Imaging Group, Division of Anaesthesia, Department of Medicine, University of Cambridge, Addenbrooke's Hospital, Cambridge CB2 0SP, United Kingdom.
Proc Natl Acad Sci U S A. 2024 Jul 23;121(30):e2320378121. doi: 10.1073/pnas.2320378121. Epub 2024 Jul 15.
The neuroscientific examination of music processing in audio-visual contexts offers a valuable framework to assess how auditory information influences the emotional encoding of visual information. Using fMRI during naturalistic film viewing, we investigated the neural mechanisms underlying the effect of music on valence inferences during mental state attribution. Thirty-eight participants watched the same short-film accompanied by systematically controlled consonant or dissonant music. Subjects were instructed to think about the main character's intentions. The results revealed that increasing levels of dissonance led to more negatively valenced inferences, displaying the profound emotional impact of musical dissonance. Crucially, at the neuroscientific level and despite music being the sole manipulation, dissonance evoked the response of the primary visual cortex (V1). Functional/effective connectivity analysis showed a stronger coupling between the auditory ventral stream (AVS) and V1 in response to tonal dissonance and demonstrated the modulation of early visual processing via top-down feedback inputs from the AVS to V1. These V1 signal changes indicate the influence of high-level contextual representations associated with tonal dissonance on early visual cortices, serving to facilitate the emotional interpretation of visual information. Our results highlight the significance of employing systematically controlled music, which can isolate emotional valence from the arousal dimension, to elucidate the brain's sound-to-meaning interface and its distributive crossmodal effects on early visual encoding during naturalistic film viewing.
视听环境下音乐加工的神经科学研究为评估听觉信息如何影响视觉信息的情感编码提供了有价值的框架。我们使用 fMRI 在自然观影条件下，研究了音乐对心理状态归因过程中效价推断的影响的神经机制。38 名参与者观看了相同的短片，同时伴有系统控制的和谐或不和谐音乐。要求参与者思考主角的意图。结果表明，不和谐程度的增加导致更负面的效价推断，显示出音乐不和谐的深刻情感影响。至关重要的是，在神经科学水平上，尽管音乐是唯一的操作变量，但不和谐会引起初级视觉皮层（V1）的反应。功能/有效连接分析显示，在对音调不和谐的反应中，听觉腹侧流（AVS）和 V1 之间的耦合更强，并证明了来自 AVS 到 V1 的自上而下反馈输入对早期视觉处理的调制。这些 V1 信号变化表明与音调不和谐相关的高级上下文表示对早期视觉皮质的影响，有助于促进对视觉信息的情感解释。我们的研究结果强调了使用系统控制音乐的重要性，这可以将情绪效价与唤醒维度分离，以阐明大脑的声音到意义接口及其在自然观影期间对早期视觉编码的分布式跨模态影响。
