Fantoni M, Federici A, Camponogara I, Handjaras G, Martinelli A, Bednaya E, Ricciardi E, Pavani F, Bottari D
MoMiLab, IMT School for Advanced Studies Lucca, Lucca, Italy.
Zayed University, Abu Dhabi, United Arab Emirates.
Heliyon. 2024 Jul 19;10(15):e34860. doi: 10.1016/j.heliyon.2024.e34860. eCollection 2024 Aug 15.
Face masks provide fundamental protection against the transmission of respiratory viruses but hamper communication. We estimated auditory and visual obstacles generated by face masks on communication by measuring the neural tracking of speech. To this end, we recorded the EEG while participants were exposed to naturalistic audio-visual speech, embedded in 5-talker noise, in three contexts: (i) no-mask (audio-visual information was fully available), (ii) virtual mask (occluded lips, but intact audio), and (iii) real mask (occluded lips and degraded audio). Neural tracking of lip movements and of the sound envelope of speech was measured through backward modeling, that is, by reconstructing stimulus properties from neural activity. Behaviorally, face masks increased perceived listening difficulty and phonological errors in speech content retrieval. At the neural level, we observed that the occlusion of the mouth abolished lip tracking and dampened neural tracking of the speech envelope at the earliest processing stages. By contrast, degraded acoustic information related to face mask filtering altered neural tracking of speech envelope at later processing stages. Finally, a consistent link emerged between the increment of perceived listening difficulty and the drop in reconstruction performance of speech envelope when attending to a speaker wearing a face mask. Results clearly dissociated the visual and auditory impact of face masks on the neural tracking of speech. While the visual obstacle related to face masks hampered the ability to predict and integrate audio-visual speech, the auditory filter generated by face masks impacted neural processing stages typically associated with auditory selective attention. The link between perceived difficulty and neural tracking drop also provides evidence of the impact of face masks on the metacognitive levels subtending face-to-face communication.
口罩能为抵御呼吸道病毒传播提供基本保护，但会妨碍交流。我们通过测量语音的神经追踪来估计口罩在交流中产生的听觉和视觉障碍。为此，我们在三种情境下记录了参与者在5人嘈杂声中接触自然视听语音时的脑电图：（i）不戴口罩（视听信息完全可用），（ii）虚拟口罩（嘴唇被遮挡，但音频完好），以及（iii）真实口罩（嘴唇被遮挡且音频质量下降）。通过反向建模，即从神经活动中重建刺激特性，来测量嘴唇运动和语音声纹的神经追踪。在行为层面，口罩增加了语音内容检索中感知到的听力困难和语音错误。在神经层面，我们观察到，在最早的处理阶段，嘴巴被遮挡会消除嘴唇追踪并减弱语音声纹的神经追踪。相比之下，与口罩过滤相关的声学信息退化在后期处理阶段改变了语音声纹的神经追踪。最后，当关注戴口罩的说话者时，感知到的听力困难增加与语音声纹重建性能下降之间出现了一致的关联。结果清楚地分离了口罩对语音神经追踪的视觉和听觉影响。虽然与口罩相关的视觉障碍妨碍了预测和整合视听语音的能力，但口罩产生的听觉过滤影响了通常与听觉选择性注意相关的神经处理阶段。感知困难与神经追踪下降之间的关联也提供了口罩对支撑面对面交流的元认知水平产生影响的证据。
