IEEE Trans Cybern. 2021 Dec;51(12):5954-5968. doi: 10.1109/TCYB.2020.2974688. Epub 2021 Dec 22.
For social robots to effectively engage in human-robot interaction (HRI), they need to be able to interpret human affective cues and to respond appropriately via display of their own emotional behavior. In this article, we present a novel multimodal emotional HRI architecture to promote natural and engaging bidirectional emotional communications between a social robot and a human user. User affect is detected using a unique combination of body language and vocal intonation, and multimodal classification is performed using a Bayesian Network. The Emotionally Expressive Robot utilizes the user's affect to determine its own emotional behavior via an innovative two-layer emotional model consisting of deliberative (hidden Markov model) and reactive (rule-based) layers. The proposed architecture has been implemented via a small humanoid robot to perform diet and fitness counseling during HRI. In order to evaluate the Emotionally Expressive Robot's effectiveness, a Neutral Robot that can detect user affects but lacks an emotional display, was also developed. A between-subjects HRI experiment was conducted with both types of robots. Extensive results have shown that both robots can effectively detect user affect during the real-time HRI. However, the Emotionally Expressive Robot can appropriately determine its own emotional response based on the situation at hand and, therefore, induce more user positive valence and less negative arousal than the Neutral Robot.
为了使社交机器人能够有效地进行人机交互（HRI），它们需要能够解释人类的情感线索，并通过展示自己的情感行为来做出适当的反应。在本文中，我们提出了一种新颖的多模态情感 HRI 架构，以促进社交机器人和人类用户之间自然而引人入胜的双向情感交流。用户的情感是通过独特的身体语言和语音语调组合来检测的，使用贝叶斯网络进行多模态分类。情感表达机器人利用用户的情感通过创新的两层情感模型来确定自己的情感行为，该模型由深思熟虑的（隐马尔可夫模型）和反应性的（基于规则的）层组成。该架构已通过一个小型人形机器人实现，以便在 HRI 期间进行饮食和健身咨询。为了评估情感表达机器人的有效性，还开发了一种能够检测用户情感但缺乏情感显示的中性机器人。使用这两种类型的机器人进行了一项被试间的 HRI 实验。广泛的结果表明，这两种机器人都可以在实时 HRI 中有效地检测用户的情感。然而，情感表达机器人可以根据当前情况适当确定自己的情感反应，因此，与中性机器人相比，它可以引起更多的用户正效价和更少的负唤醒。
