The Cross-Modal Suppressive Role of Visual Context on Speech Intelligibility: An ERP Study.

The efficacy of audiovisual (AV) integration is reflected in the degree of cross-modal suppression of the auditory event-related potentials (ERPs, P1-N1-P2), while stronger semantic encoding is reflected in enhanced late ERP negativities (e.g., N450). We hypothesized that increasing visual stimulus reliability should lead to more robust AV-integration and enhanced semantic prediction, reflected in suppression of auditory ERPs and enhanced N450, respectively. EEG was acquired while individuals watched and listened to and videos of a speaker uttering or highly-intelligible degraded () words and made binary judgments about word meaning (animate or inanimate). We found that speech evoked larger negativity between 280-527-ms than speech, suggestive of more robust semantic prediction for the signal. For visual reliability, we found that greater cross-modal ERP suppression occurred for than videos prior to sound onset and for the P2 ERP. Additionally, the later semantic-related negativity tended to be larger for than videos. These results suggest that the cross-modal effect is largely confined to suppression of early auditory networks with weak effect on networks associated with semantic prediction. However, the semantic-related visual effect on the late negativity may have been tempered by the vocoded signal's high-reliability.