Shahin Antoine J, Shen Stanley, Kerlin Jess R
Center for Mind and Brain, University of California, Davis, CA, 95618.
Lang Cogn Neurosci. 2017;32(9):1102-1118. doi: 10.1080/23273798.2017.1283428. Epub 2017 Feb 6.
We examined the relationship between tolerance for audiovisual onset asynchrony (AVOA) and the spectrotemporal fidelity of the spoken words and the speaker's mouth movements. In two experiments that only varied in the temporal order of sensory modality, visual speech leading (exp1) or lagging (exp2) acoustic speech, participants watched intact and blurred videos of a speaker uttering trisyllabic words and nonwords that were noise vocoded with 4-, 8-, 16-, and 32-channels. They judged whether the speaker's mouth movements and the speech sounds were or . Individuals perceived synchrony (tolerated AVOA) on more trials when the acoustic speech was more speech-like (8 channels and higher vs. 4 channels), and when visual speech was intact than blurred (exp1 only). These findings suggest that enhanced spectrotemporal fidelity of the audiovisual (AV) signal prompts the brain to widen the window of integration promoting the fusion of temporally distant AV percepts.
我们研究了视听起始异步性(AVOA)耐受性与口语单词的频谱时间保真度以及说话者口部动作之间的关系。在两个仅在感觉模态时间顺序上有所不同的实验中,视觉语音领先(实验1)或滞后(实验2)声学语音,参与者观看了一位说话者说出三音节单词和非单词的完整和模糊视频,这些视频经过4、8、16和32通道的噪声声码处理。他们判断说话者的口部动作和语音是否同步。当声学语音更像语音时(8通道及以上与4通道相比),以及当视觉语音完整而非模糊时(仅实验1),个体在更多试验中感知到同步(耐受AVOA)。这些发现表明,视听(AV)信号增强的频谱时间保真度促使大脑扩大整合窗口,促进时间上遥远的AV感知的融合。