Gast Volker
Department of English and American Studies, Friedrich Schiller University, 07743 Jena, Germany.
Behav Sci (Basel). 2023 Jan 6;13(1):52. doi: 10.3390/bs13010052.
Previous research has shown that eyebrow movement during speech exhibits a systematic relationship with intonation: brow raises tend to be aligned with pitch accents, typically preceding them. The present study approaches the question of temporal alignment between brow movement and intonation from a new angle. The study makes use of footage from the , processed with 3D facial landmark detection. Pitch is modeled as a sinusoidal function whose parameters are correlated with the maximum height of the eyebrows in a brow raise. The results confirm some previous findings on audiovisual prosody but lead to new insights as well. First, the shape of the pitch signal in a region of approx. 630 ms before the brow raise is not random and tends to display a specific shape. Second, while being less informative than the post-peak pitch, the pitch signal in the pre-peak region also exhibits correlations with the magnitude of the associated brow raises. Both of these results point to early preparatory action in the speech signal, calling into question the visual-precedes-acoustic assumption. The results are interpreted as supporting a unified view of gesture/speech co-production that regards both signals as manifestations of a single communicative act.
先前的研究表明,说话时眉毛的运动与语调呈现出一种系统的关系:眉毛上扬往往与音高重音对齐,通常在音高重音之前。本研究从一个新的角度探讨了眉毛运动与语调之间的时间对齐问题。该研究利用了[具体来源未提及]的视频片段,并通过3D面部地标检测进行处理。音高被建模为一个正弦函数,其参数与眉毛上扬时的最大高度相关。研究结果证实了先前关于视听韵律的一些发现,但也带来了新的见解。首先,在眉毛上扬前约630毫秒的区域内,音高信号的形状并非随机,而是倾向于呈现出一种特定的形状。其次,虽然峰值前的音高信号比峰值后的音高信号信息量少,但它也与相关眉毛上扬的幅度存在相关性。这两个结果都表明语音信号中存在早期的准备动作,这对视觉先于听觉的假设提出了质疑。这些结果被解释为支持一种手势/语音协同产生的统一观点,即认为这两种信号都是单一交际行为的表现形式。