Department of Basic Neurosciences, Faculty of Medicine, University of Geneva, 1202 Geneva, Switzerland.
Division of Neurology, Department of Clinical Neurosciences, Geneva University Hospitals, 1205 Geneva, Switzerland.
Sci Adv. 2020 Nov 4;6(45). doi: 10.1126/sciadv.abc6348. Print 2020 Nov.
When we see our interlocutor, our brain seamlessly extracts visual cues from their face and processes them along with the sound of their voice, making speech an intrinsically multimodal signal. Visual cues are especially important in noisy environments, when the auditory signal is less reliable. Neuronal oscillations might be involved in the cortical processing of audiovisual speech by selecting which sensory channel contributes more to perception. To test this, we designed computer-generated naturalistic audiovisual speech stimuli where one mismatched phoneme-viseme pair in a key word of sentences created bistable perception. Neurophysiological recordings (high-density scalp and intracranial electroencephalography) revealed that the precise phase angle of theta-band oscillations in posterior temporal and occipital cortex of the right hemisphere was crucial to select whether the auditory or the visual speech cue drove perception. We demonstrate that the phase of cortical oscillations acts as an instrument for sensory selection in audiovisual speech processing.
当我们看到对话者时，大脑会从他们的面部无缝提取视觉线索，并与声音一起处理这些线索，使言语成为一种内在的多模态信号。在嘈杂的环境中，听觉信号不太可靠时，视觉线索尤为重要。神经元振荡可能参与了视听言语的皮层处理，选择哪个感觉通道对感知的贡献更大。为了验证这一点，我们设计了计算机生成的自然视听言语刺激，在句子的关键词中，一个不匹配的音位-视位对会产生双稳态感知。神经生理学记录（高密度头皮和颅内脑电图）显示，右半球后颞叶和枕叶皮质中θ带振荡的精确相位角对于选择听觉还是视觉言语线索驱动感知至关重要。我们证明，皮层振荡的相位在视听言语处理中充当了感觉选择的工具。
