Zhuang Meiqi, Yin Lang, Wang Youhua, Bai Yunzhao, Zhan Jian, Hou Chao, Yin Liting, Xu Zhangyu, Tan Xiaohui, Huang YongAn
Information Engineering College, Capital Normal University, Beijing 100048, China.
State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China.
Research (Wash D C). 2021 Jul 15;2021:9759601. doi: 10.34133/2021/9759601. eCollection 2021.
The facial expressions are a mirror of the elusive emotion hidden in the mind, and thus, capturing expressions is a crucial way of merging the inward world and virtual world. However, typical facial expression recognition (FER) systems are restricted by environments where faces must be clearly seen for computer vision, or rigid devices that are not suitable for the time-dynamic, curvilinear faces. Here, we present a robust, highly wearable FER system that is based on deep-learning-assisted, soft epidermal electronics. The epidermal electronics that can fully conform on faces enable high-fidelity biosignal acquisition without hindering spontaneous facial expressions, releasing the constraint of movement, space, and light. The deep learning method can significantly enhance the recognition accuracy of facial expression types and intensities based on a small sample. The proposed wearable FER system is superior for wide applicability and high accuracy. The FER system is suitable for the individual and shows essential robustness to different light, occlusion, and various face poses. It is totally different from but complementary to the computer vision technology that is merely suitable for simultaneous FER of multiple individuals in a specific place. This wearable FER system is successfully applied to human-avatar emotion interaction and verbal communication disambiguation in a real-life environment, enabling promising human-computer interaction applications.
面部表情是隐藏在内心难以捉摸的情感的一面镜子，因此，捕捉表情是融合内心世界和虚拟世界的关键方式。然而，典型的面部表情识别（FER）系统受到环境的限制，在这些环境中，计算机视觉必须清晰地看到面部，或者受到不适合随时间动态变化、呈曲线状面部的刚性设备的限制。在此，我们展示了一种基于深度学习辅助的柔性表皮电子器件的强大、高度可穿戴的FER系统。能够完全贴合面部的表皮电子器件可实现高保真生物信号采集，同时不妨碍自然的面部表情，解除了运动、空间和光线的限制。深度学习方法能够基于少量样本显著提高面部表情类型和强度的识别准确率。所提出的可穿戴FER系统在广泛适用性和高精度方面表现出色。该FER系统适用于个体，并且对不同的光线、遮挡和各种面部姿势具有基本的鲁棒性。它与仅适用于在特定场所同时对多个人进行FER的计算机视觉技术完全不同但又相互补充。这种可穿戴FER系统已成功应用于现实生活环境中的人机情感交互和言语交流消歧，为有前景的人机交互应用提供了可能。
