Starkey Hearing Research Center, Berkeley, California, USA.
Ear Hear. 2011 Sep-Oct;32(5):582-92. doi: 10.1097/AUD.0b013e31820fca23.
The objective of this study is to explore the sensitivity to intermodal asynchrony in audiovisual speech with moderate to severe sensorineural hearing loss. Based on previous studies, two opposing expectations were an increase in sensitivity, as hearing-impaired listeners heavily rely on lipreading in daily life, and a reduction in sensitivity, as hearing-impaired listeners tend to be elderly and advanced age could potentially impair audiovisual integration.
Adults with normal (N = 11, ages between 23 and 50 yrs) and impaired hearing (N = 11, ages between 54 and 81 yrs, the pure-tone average between 42 and 67 dB HL) participated in two experiments. In the first experiment, the synchrony judgments were recorded for varying intermodal time differences in audiovisual sentence recordings. In the second experiment, the intelligibility of audiovisual and audio-only speech was measured in speech-shaped noise, and correlations were explored between the synchrony window and intelligibility scores for individual listeners.
Similar to previous studies, a sensitivity window on the order of a few hundred milliseconds was observed with all listeners. The average window shapes did not differ between normal-hearing and hearing-impaired groups; however, there was large individual variability. Individual windows were quantified by Gaussian curve fitting. Point of subjective simultaneity, a measure of window peak shift from the actual synchrony point, and full-width at half-maximum, a measure of window duration, were not correlated with participant's age or the degree of hearing loss. Points of subjective simultaneity were also not correlated with speech intelligibility scores. A moderate negative correlation that was significant at most conditions was observed between the full-width at half-maximum values and intelligibility scores.
Contrary to either expectation per se, there was no indication of an effect of hearing impairment or age on the sensitivity to intermodal asynchrony in audiovisual speech. It is possible that the negative effects of aging were balanced with the positive effects of increased sensitivity due to reliance on visual cues with hearing impairment. The listeners, normal hearing or hearing impaired, who were more sensitive to asynchrony (with narrower synchrony windows) tended to understand speech in noise better, with both audio-only and audiovisual speech. The practical implication of the results is that delays in audio or video signals of communication systems would affect hearing-impaired listeners in a manner similar to normal-hearing listeners, and due to the importance of visual cues for the hearing-impaired listeners, special attention should be given to limit these delays.
本研究旨在探讨中重度感音神经性听力损失患者对视听语音的多模态失同步敏感性。基于以往的研究，有两种相反的预期：一种是敏感性增加，因为听力受损的听众在日常生活中严重依赖唇读；另一种是敏感性降低，因为听力受损的听众往往年龄较大，而年龄增长可能会损害视听整合。
正常听力（N=11，年龄 23-50 岁）和听力受损（N=11，年龄 54-81 岁，纯音平均听阈在 42-67dB HL）的成年人参加了两个实验。在第一个实验中，记录了视听句子记录中不同的多模态时间差异的同步判断。在第二个实验中，在语音噪声中测量了视听和仅听语音的可懂度，并对个体听众的同步窗口和可懂度得分进行了相关性探索。
与以往的研究类似，所有听众都观察到了大约几百毫秒的敏感性窗口。正常听力和听力受损组的平均窗口形状没有差异；然而，个体之间存在很大的差异。通过高斯曲线拟合来量化个体窗口。主观同时性的点，即窗口峰值从实际同步点的偏移量的测量值，以及半最大值全宽，即窗口持续时间的测量值，与参与者的年龄或听力损失程度无关。主观同时性的点也与言语可懂度得分无关。在大多数情况下，观察到半最大值全宽值与可懂度得分之间存在中度负相关，且相关性显著。
与听力损伤或年龄的影响本身相反，在视听语音的多模态失同步敏感性方面，没有迹象表明听力损伤或年龄有影响。由于听力损伤患者依赖视觉线索，因此年龄增长的负面影响可能与由于依赖视觉线索而导致的敏感性增加的积极影响相平衡。对失同步更敏感（同步窗口较窄）的听众，无论是正常听力还是听力受损，对语音噪声中的言语理解都更好，无论是仅听音频还是视听语音。结果的实际意义是，通信系统中音频或视频信号的延迟将以与正常听力听众相似的方式影响听力受损的听众，并且由于视觉线索对听力受损听众的重要性，应特别注意限制这些延迟。
