Institute of Research in Psychology (IPSY), Institute of Neuroscience (IoNS), University of Louvain (UCL), Louvain-la-Neuve, Belgium.
Institute of Research in Psychology (IPSY), Institute of Neuroscience (IoNS), University of Louvain (UCL), Louvain-la-Neuve, Belgium; Centre for Mind/Brain Sciences, University of Trento, Trento, Italy.
Cortex. 2019 Oct;119:184-194. doi: 10.1016/j.cortex.2019.04.017. Epub 2019 May 6.
Humans seamlessly extract and integrate the emotional content delivered by the face and the voice of others. It is however poorly understood how perceptual decisions unfold in time when people discriminate the expression of emotions transmitted using dynamic facial and vocal signals, as in natural social context. In this study, we relied on a gating paradigm to track how the recognition of emotion expressions across the senses unfold over exposure time. We first demonstrate that across all emotions tested, a discriminatory decision is reached earlier with faces than with voices. Importantly, multisensory stimulation consistently reduced the required accumulation of perceptual evidences needed to reach correct discrimination (Isolation Point). We also observed that expressions with different emotional content provide cumulative evidence at different speeds, with "fear" being the expression with the fastest isolation point across the senses. Finally, the lack of correlation between the confusion patterns in response to facial and vocal signals across time suggest distinct relations between the discriminative features extracted from the two signals. Altogether, these results provide a comprehensive view on how auditory, visual and audiovisual information related to different emotion expressions accumulate in time, highlighting how multisensory context can fasten the discrimination process when minimal information is available.
人类能够轻松地提取和整合他人面部表情和声音所传递的情感内容。然而，当人们在自然社交环境中通过动态面部和语音信号来辨别情绪表达时，人们对于感知决策如何随时间展开还知之甚少。在这项研究中，我们依赖门控范式来追踪跨感觉的情绪表达识别是如何随时间展开的。我们首先证明，在所有测试的情绪中，面孔比声音更早做出辨别决策。重要的是，多感官刺激始终会减少达到正确辨别所需的感知证据的积累（隔离点）。我们还观察到，具有不同情感内容的表情以不同的速度提供累积证据，而“恐惧”是跨感觉的最快隔离点的表情。最后，面部和语音信号在时间上的反应混淆模式之间缺乏相关性表明，从两个信号中提取的判别特征之间存在不同的关系。总之，这些结果全面展示了与不同情绪表达相关的听觉、视觉和视听信息如何随时间累积，强调了在可用信息最少的情况下，多感官环境如何能够加快辨别过程。
