Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, The Netherlands.
Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525XD, Nijmegen, The Netherlands.
Sci Rep. 2021 Aug 18;11(1):16721. doi: 10.1038/s41598-021-95791-0.
In everyday conversation, we are often challenged with communicating in non-ideal settings, such as in noise. Increased speech intensity and larger mouth movements are used to overcome noise in constrained settings (the Lombard effect). How we adapt to noise in face-to-face interaction, the natural environment of human language use, where manual gestures are ubiquitous, is currently unknown. We asked Dutch adults to wear headphones with varying levels of multi-talker babble while attempting to communicate action verbs to one another. Using quantitative motion capture and acoustic analyses, we found that (1) noise is associated with increased speech intensity and enhanced gesture kinematics and mouth movements, and (2) acoustic modulation only occurs when gestures are not present, while kinematic modulation occurs regardless of co-occurring speech. Thus, in face-to-face encounters the Lombard effect is not constrained to speech but is a multimodal phenomenon where the visual channel carries most of the communicative burden.
在日常对话中，我们经常面临在不理想的环境中进行交流的挑战，例如在噪音环境中。为了克服受限环境中的噪音，人们会提高说话的音量并加大嘴部动作（伦巴第效应）。然而，我们目前还不清楚在人类语言使用的自然环境——面对面互动中，人们如何适应噪音。我们要求荷兰成年人在佩戴耳机并播放多说话人背景噪声的情况下，彼此尝试用动作动词进行交流。通过使用定量运动捕捉和声学分析，我们发现：（1）噪声与提高的语音强度以及增强的手势运动学和嘴部动作有关；（2）只有在没有手势时才会出现声学调制，而运动学调制则无论是否存在语音都会发生。因此，在面对面交流中，伦巴第效应不仅限于语音，而是一种多模态现象，其中视觉通道承担了大部分的交流负担。
