Department of Speech, Language, and Hearing Sciences, University of Florida, Gainesville, Florida 32610, USA.
Department of Linguistics, University of Florida, Gainesville, Florida 32611,
J Acoust Soc Am. 2020 Mar;147(3):EL246. doi: 10.1121/10.0000737.
The nature of the visual input that integrates with the audio signal to yield speech processing advantages remains controversial. This study tests the hypothesis that the information extracted for audiovisual integration includes co-occurring suprasegmental dynamic changes in the acoustic and visual signal. English sentences embedded in multi-talker babble noise were presented to native English listeners in audio-only and audiovisual modalities. A significant intelligibility enhancement with the visual analogs congruent to the acoustic amplitude envelopes was observed. These results suggest that dynamic visual modulation provides speech rhythmic information that can be integrated online with the audio signal to enhance speech intelligibility.
视觉输入的本质与音频信号相结合，产生了言语处理优势，这一问题仍存在争议。本研究检验了这样一个假设，即用于视听整合的信息包括声学和视觉信号中同时发生的超音段动态变化。将嵌入多说话人背景噪声的英语句子仅以音频和视听两种方式呈现给以英语为母语的听众。观察到与声学幅度包络一致的视觉模拟有显著的可懂度增强。这些结果表明，动态视觉调制提供了言语节奏信息，可以与音频信号在线整合，从而提高言语可懂度。
