Suppr超能文献

言语伴随的眉毛运动与音高的时间对齐:一项基于深夜脱口秀访谈的研究

The Temporal Alignment of Speech-Accompanying Eyebrow Movement and Voice Pitch: A Study Based on Late Night Show Interviews.

作者信息

Gast Volker

机构信息

Department of English and American Studies, Friedrich Schiller University, 07743 Jena, Germany.

出版信息

Behav Sci (Basel). 2023 Jan 6;13(1):52. doi: 10.3390/bs13010052.

Abstract

Previous research has shown that eyebrow movement during speech exhibits a systematic relationship with intonation: brow raises tend to be aligned with pitch accents, typically preceding them. The present study approaches the question of temporal alignment between brow movement and intonation from a new angle. The study makes use of footage from the , processed with 3D facial landmark detection. Pitch is modeled as a sinusoidal function whose parameters are correlated with the maximum height of the eyebrows in a brow raise. The results confirm some previous findings on audiovisual prosody but lead to new insights as well. First, the shape of the pitch signal in a region of approx. 630 ms before the brow raise is not random and tends to display a specific shape. Second, while being less informative than the post-peak pitch, the pitch signal in the pre-peak region also exhibits correlations with the magnitude of the associated brow raises. Both of these results point to early preparatory action in the speech signal, calling into question the visual-precedes-acoustic assumption. The results are interpreted as supporting a unified view of gesture/speech co-production that regards both signals as manifestations of a single communicative act.

摘要

先前的研究表明,说话时眉毛的运动与语调呈现出一种系统的关系:眉毛上扬往往与音高重音对齐,通常在音高重音之前。本研究从一个新的角度探讨了眉毛运动与语调之间的时间对齐问题。该研究利用了[具体来源未提及]的视频片段,并通过3D面部地标检测进行处理。音高被建模为一个正弦函数,其参数与眉毛上扬时的最大高度相关。研究结果证实了先前关于视听韵律的一些发现,但也带来了新的见解。首先,在眉毛上扬前约630毫秒的区域内,音高信号的形状并非随机,而是倾向于呈现出一种特定的形状。其次,虽然峰值前的音高信号比峰值后的音高信号信息量少,但它也与相关眉毛上扬的幅度存在相关性。这两个结果都表明语音信号中存在早期的准备动作,这对视觉先于听觉的假设提出了质疑。这些结果被解释为支持一种手势/语音协同产生的统一观点,即认为这两种信号都是单一交际行为的表现形式。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/30a8/9854528/a3c619a5bda8/behavsci-13-00052-g0A1.jpg

相似文献

2
Prosodic structure shapes the temporal realization of intonation and manual gesture movements.
J Speech Lang Hear Res. 2013 Jun;56(3):850-64. doi: 10.1044/1092-4388(2012/12-0049). Epub 2012 Dec 28.
3
Eyebrow movements and vocal pitch height: evidence consistent with an ethological signal.
J Acoust Soc Am. 2013 May;133(5):2947-52. doi: 10.1121/1.4798801.
4
Eyebrow peak position redefined.
Aesthet Surg J. 2010 May-Jun;30(3):297-300. doi: 10.1177/1090820X10369918.
5
Postfocal Downstep in German.
Lang Speech. 2017 Jun;60(2):260-288. doi: 10.1177/0023830916647204. Epub 2016 Jun 1.
7
Rethinking the frequency code: a meta-analytic review of the role of acoustic body size in communicative phenomena.
Philos Trans R Soc Lond B Biol Sci. 2021 Dec 20;376(1840):20200400. doi: 10.1098/rstb.2020.0400. Epub 2021 Nov 1.
9
The Prosodic Characteristics of Non-referential Co-speech Gestures in a Sample of Academic-Lecture-Style Speech.
Front Psychol. 2018 Sep 7;9:1514. doi: 10.3389/fpsyg.2018.01514. eCollection 2018.
10
On the intonation of sinusoidal sentences: contour and pitch height.
J Acoust Soc Am. 1993 Oct;94(4):1983-8. doi: 10.1121/1.407501.

引用本文的文献

1
Deliberate synchronization of speech and gesture: Effects of neurodiversity and development.
Lang Cogn. 2024 Dec;16(4):1812-1833. doi: 10.1017/langcog.2024.33. Epub 2024 Sep 24.

本文引用的文献

1
Prosodic Prominence - A Cross-Linguistic Perspective.
Lang Speech. 2021 Jun;64(2):253-260. doi: 10.1177/00238309211015768.
2
Gesture and Sign: Cataclysmic Break or Dynamic Relations?
Front Psychol. 2018 Sep 10;9:1651. doi: 10.3389/fpsyg.2018.01651. eCollection 2018.
3
Surprise: unfolding of facial expressions.
Cogn Emot. 2019 Aug;33(5):915-930. doi: 10.1080/02699931.2018.1517730. Epub 2018 Sep 6.
4
Gesture, sign, and language: The coming of age of sign language and gesture studies.
Behav Brain Sci. 2017 Jan;40:e46. doi: 10.1017/S0140525X15001247. Epub 2015 Oct 5.
6
BODY POSITION, FACIAL EXPRESSION, AND VERBAL BEHAVIOR DURING INTERVIEWS.
J Abnorm Psychol. 1964 Mar;68:295-301. doi: 10.1037/h0040225.
7
Pan-cultural elements in facial displays of emotion.
Science. 1969 Apr 4;164(3875):86-8. doi: 10.1126/science.164.3875.86.
8
Head and body cues in the judgment of emotion: a reformulation.
Percept Mot Skills. 1967 Jun;24(3):711-24. doi: 10.2466/pms.1967.24.3.711.

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验