Pouw Wim, de Jonge-Hoekstra Lisette, Harrison Steven J, Paxton Alexandra, Dixon James A
Center for the Ecological Study of Perception and Action, University of Connecticut, Storrs, Connecticut.
Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, the Netherlands.
Ann N Y Acad Sci. 2021 May;1491(1):89-105. doi: 10.1111/nyas.14532. Epub 2020 Dec 18.
It is commonly understood that hand gesture and speech coordination in humans is culturally and cognitively acquired, rather than having a biological basis. Recently, however, the biomechanical physical coupling of arm movements to speech vocalization has been studied in steady-state vocalization and monosyllabic utterances, where forces produced during gesturing are transferred onto the tensioned body, leading to changes in respiratory-related activity and thereby affecting vocalization F0 and intensity. In the current experiment (n = 37), we extend this previous line of work to show that gesture-speech physics also impacts fluent speech. Compared with nonmovement, participants who are producing fluent self-formulated speech while rhythmically moving their limbs demonstrate heightened F0 and amplitude envelope, and such effects are more pronounced for higher-impulse arm versus lower-impulse wrist movement. We replicate that acoustic peaks arise especially during moments of peak impulse (i.e., the beat) of the movement, namely around deceleration phases of the movement. Finally, higher deceleration rates of higher-mass arm movements were related to higher peaks in acoustics. These results confirm a role for physical impulses of gesture affecting the speech system. We discuss the implications of gesture-speech physics for understanding of the emergence of communicative gesture, both ontogenetically and phylogenetically.
人们普遍认为,人类的手势与言语协调是在文化和认知层面习得的,而非基于生物学基础。然而,最近针对手臂动作与言语发声的生物力学物理耦合,在稳态发声和单音节发声方面展开了研究,研究发现,手势过程中产生的力会传递到绷紧的身体上,进而导致与呼吸相关的活动发生变化,从而影响发声的基频(F0)和强度。在当前的实验中(n = 37),我们拓展了此前的这一系列研究工作,以表明手势 - 言语物理学也会影响流畅言语。与不做动作相比,在有节奏地移动四肢时进行流畅的自我组织言语表达的参与者,其基频和振幅包络会升高,而且相较于低冲量的手腕动作,高冲量的手臂动作产生的这种影响更为明显。我们再次验证了,声学峰值尤其会在动作的峰值冲量时刻(即节拍)出现,也就是在动作的减速阶段左右。最后,质量更大的手臂动作的更高减速速率与声学上的更高峰值相关。这些结果证实了手势的物理冲量对言语系统的影响作用。我们讨论了手势 - 言语物理学对于从个体发生和系统发生角度理解交际手势出现的意义。