Department of Linguistic, Literary, and Aesthetic Studies, University of Bergen, Bergen, Norway.
Department of Robotics and Mechatronics, School of Engineering and Digital Sciences, Nazarbayev University, Nur-Sultan, Kazakhstan.
PLoS One. 2020 Jun 2;15(6):e0233731. doi: 10.1371/journal.pone.0233731. eCollection 2020.
Facial expressions in sign languages are used to express grammatical functions, such as question marking, but can also be used to express emotions (either the signer's own or in constructed action contexts). Emotions and grammatical functions can utilize the same articulators, and the combinations can be congruent or incongruent. For instance, surprise and polar questions can be marked by raised eyebrows, while anger is usually marked by lowered eyebrows. We investigated what happens when different emotions (neutral/surprise/anger) are combined with different sentence types (statement/polar question/wh-question) in Kazakh-Russian Sign Language (KRSL), replicating studies previously made for other sign languages. We asked 9 native signers (5 deaf, 4 hearing children of deaf adults) to sign 10 simple sentences in 9 conditions (3 emotions * 3 sentence types). We used OpenPose software to track eyebrow position in the video recordings. We found that emotions and sentence types influence eyebrow position in KRSL: eyebrows are raised for polar questions and surprise, and lowered for anger. There are also some interactions between the two factors, as well as some differences between hearing and deaf native signers, namely a smaller effect of polar questions for the deaf group, and a different interaction between emotions and wh-question marking in the two groups. We thus find evidence for the complex influences on non-manual behavior in signers of sign languages, and showcase a quantitative approach to this field.
手语中的面部表情用于表达语法功能,如问号,但也可用于表达情感(无论是手语使用者自己的情感还是在构建的动作情境中)。情感和语法功能可以使用相同的发音器官,并且组合可以是一致的或不一致的。例如,惊讶和疑问可以通过挑眉来标记,而愤怒通常通过降低眉毛来标记。我们研究了在哈萨克-俄罗斯手语(KRSL)中,不同情感(中性/惊讶/愤怒)与不同句子类型(陈述句/疑问/特殊疑问句)结合时会发生什么,复制了之前针对其他手语进行的研究。我们要求 9 名母语为手语的人(5 名聋人,4 名聋人父母的听力正常儿童)在 9 种条件下(3 种情感*3 种句子类型)用 KRSL 签署 10 个简单句子。我们使用 OpenPose 软件来跟踪视频记录中的眉毛位置。我们发现情绪和句子类型会影响 KRSL 中的眉毛位置:疑问句和惊讶时眉毛上扬,愤怒时眉毛下降。这两个因素之间也存在一些相互作用,以及听力和失聪的母语使用者之间的一些差异,即对于失聪组,疑问句的影响较小,并且两组之间的情感和特殊疑问句标记之间的相互作用不同。因此,我们在手语使用者的非语言行为方面发现了复杂影响的证据,并展示了一种针对该领域的定量方法。