Donders Institute for Brain, Cognition and Behaviour, Radboud University.
Max Planck Institute for Psycholinguistics.
Cogn Sci. 2023 Jun;47(6):e13298. doi: 10.1111/cogs.13298.
In conversation, individuals work together to achieve communicative goals, complementing and aligning language and body with each other. An important emerging question is whether interlocutors entrain with one another equally across linguistic levels (e.g., lexical, syntactic, and semantic) and modalities (i.e., speech and gesture), or whether there are complementary patterns of behaviors, with some levels or modalities diverging and others converging in coordinated fashions. This study assesses how kinematic and linguistic entrainment interact with one another across levels of measurement, and according to communicative context. We analyzed data from two matched corpora of dyadic interaction between-respectively-Danish and Norwegian native speakers engaged in affiliative conversations and task-oriented conversations. We assessed linguistic entrainment at the lexical, syntactic, and semantic level, and kinetic alignment of the head and hands using video-based motion tracking and dynamic time warping. We tested whether-across the two languages-linguistic alignment correlates with kinetic alignment, and whether these kinetic-linguistic associations are modulated either by the type of conversation or by the language spoken. We found that kinetic entrainment was positively associated with low-level linguistic (i.e., lexical) entrainment, while negatively associated with high-level linguistic (i.e., semantic) entrainment, in a cross-linguistically robust way. Our findings suggest that conversation makes use of a dynamic coordination of similarity and complementarity both between individuals as well as between different communicative modalities, and provides evidence for a multimodal, interpersonal synergy account of interaction.
在会话中,个体通过相互配合来实现交际目标,使语言和身体彼此补充并协调一致。一个新兴的重要问题是,对话者是否在语言层面(如词汇、句法和语义)和模态(即言语和手势)上平等地相互协调,或者是否存在互补的行为模式,某些层面或模态存在分歧,而其他层面或模态则以协调的方式趋同。本研究评估了在不同的测量水平和交际语境下,运动和语言的同步相互作用。我们分析了来自两个分别由丹麦语和挪威语母语者进行的、针对亲和性对话和任务导向性对话的配对语料库的数据。我们评估了词汇、句法和语义层面的语言同步,以及使用基于视频的运动跟踪和动态时间规整的头部和手部的运动同步。我们检验了在这两种语言中,语言同步是否与运动同步相关,以及这些运动-语言的关联是否受到对话类型或使用的语言的影响。我们发现,运动同步与低水平的语言(即词汇)同步呈正相关,而与高水平的语言(即语义)同步呈负相关,这在跨语言上是稳健的。我们的研究结果表明,会话在个体之间以及不同交际模态之间利用了相似性和互补性的动态协调,并为互动的多模态、人际协同作用提供了证据。