Rilliard Albert, Shochi Takaaki, Martin Jean-Claude, Erickson Donna, Aubergé Véronique
LIMSI-CNRS, Orsay, France.
Lang Speech. 2009;52(Pt 2-3):223-43. doi: 10.1177/0023830909103171.
Whereas several studies have explored the expression of emotions, little is known on how the visual and audio channels are combined during production of what we call the more controlled social affects, for example, "attitudinal" expressions. This article presents a perception study of the audovisual expression of 12 Japanese and 6 French attitudes in order to understand the contribution of audio and visual modalities for affective communication. The relative importance of each modality in the perceptual decoding of the expressions of four speakers is analyzed as a first step towards a deeper comprehension of their influence on the expression of social affects. Then, the audovisual productions of two speakers (one for each language) are acoustically (F0, duration and intensity) and visually (in terms of Action Units) analyzed, in order to match the relation between objective parameters and listeners' perception of these social affects. The most pertinent objective features, either acoustic or visual, are then discussed, in a bilingual perspective: for example, the relative influence of fundamental frequency for attitudinal expression in both languages is discussed, and the importance of a certain aspect of the voice quality dimension in Japanese is underlined.
尽管已有多项研究探讨了情感表达，但对于在产生我们所谓的更具可控性的社会情感（例如“态度”表达）过程中视觉和听觉通道是如何结合的，我们却知之甚少。本文呈现了一项关于12种日语态度和6种法语态度的视听表达的感知研究，旨在了解听觉和视觉模态在情感交流中的作用。作为深入理解它们对社会情感表达影响的第一步，分析了每种模态在四位说话者表达的感知解码中的相对重要性。然后，对两位说话者（每种语言各一位）的视听作品进行声学（基频、时长和强度）和视觉（从动作单元角度）分析，以匹配客观参数与听众对这些社会情感的感知之间的关系。接着，从双语角度讨论最相关的客观特征，无论是声学特征还是视觉特征：例如，讨论了基频在两种语言中对态度表达的相对影响，并强调了日语中语音质量维度某一方面的重要性。
