Creative Machines Laboratory, Department of Mechanical Engineering, Columbia University, New York, NY 10027, USA.
Mechanical Engineering and Materials Department, Duke University, Durham, NC 27708, USA.
Sci Robot. 2024 Mar 27;9(88):eadi4724. doi: 10.1126/scirobotics.adi4724.
Large language models are enabling rapid progress in robotic verbal communication, but nonverbal communication is not keeping pace. Physical humanoid robots struggle to express and communicate using facial movement, relying primarily on voice. The challenge is twofold: First, the actuation of an expressively versatile robotic face is mechanically challenging. A second challenge is knowing what expression to generate so that the robot appears natural, timely, and genuine. Here, we propose that both barriers can be alleviated by training a robot to anticipate future facial expressions and execute them simultaneously with a human. Whereas delayed facial mimicry looks disingenuous, facial coexpression feels more genuine because it requires correct inference of the human's emotional state for timely execution. We found that a robot can learn to predict a forthcoming smile about 839 milliseconds before the human smiles and, using a learned inverse kinematic facial self-model, coexpress the smile simultaneously with the human. We demonstrated this ability using a robot face comprising 26 degrees of freedom. We believe that the ability to coexpress simultaneous facial expressions could improve human-robot interaction.
大型语言模型正在推动机器人言语交流的快速发展，但非言语交流却没有跟上步伐。人形机器人在表达和交流方面主要依赖于声音，在使用面部运动方面还存在困难。这一挑战有两个方面：首先，表达性强的机器人面部的驱动机制在机械上具有挑战性。其次，要知道生成什么样的表情才能让机器人看起来自然、及时和真实。在这里，我们提出，通过训练机器人来预测未来的面部表情，并与人类同步执行这些表情，可以缓解这两个障碍。虽然延迟的面部模仿看起来不真诚，但面部共表达因为需要正确推断人类的情绪状态才能及时执行，所以感觉更真实。我们发现，机器人可以学会在人类微笑前约 839 毫秒预测到即将到来的微笑，并且可以使用学习到的逆向运动学面部自模型与人类同步表达微笑。我们使用了一个包含 26 个自由度的机器人面部来展示这种能力。我们相信，同步表达面部表情的能力可以改善人机交互。
