German Research Center for Artificial Intelligence(DFKI), 67663 Kaiserslautern, Germany.
Department of Computer Science, University of Kaiserslautern, 67663 Kaiserslautern, Germany.
Sensors (Basel). 2020 Aug 30;20(17):4904. doi: 10.3390/s20174904.
Many human activities and states are related to the facial muscles' actions: from the expression of emotions, stress, and non-verbal communication through health-related actions, such as coughing and sneezing to nutrition and drinking. In this work, we describe, in detail, the design and evaluation of a wearable system for facial muscle activity monitoring based on a re-configurable differential array of stethoscope-microphones. In our system, six stethoscopes are placed at locations that could easily be integrated into the frame of smart glasses. The paper describes the detailed hardware design and selection and adaptation of appropriate signal processing and machine learning methods. For the evaluation, we asked eight participants to imitate a set of facial actions, such as expressions of happiness, anger, surprise, sadness, upset, and disgust, and gestures, like kissing, winkling, sticking the tongue out, and taking a pill. An evaluation of a complete data set of 2640 events with 66% training and a 33% testing rate has been performed. Although we encountered high variability of the volunteers' expressions, our approach shows a recall = 55%, precision = 56%, and f1-score of 54% for the user-independent scenario(9% chance-level). On a user-dependent basis, our worst result has an f1-score = 60% and best result with f1-score = 89%. Having a recall ≥60% for expressions like happiness, anger, kissing, sticking the tongue out, and neutral(Null-class).
从表达情绪、压力和非言语交流，到与健康相关的动作，如咳嗽和打喷嚏，再到营养和饮水。在这项工作中，我们详细描述了一种基于听诊器麦克风可重构差分阵列的面部肌肉活动监测可穿戴系统的设计和评估。在我们的系统中，六个听诊器放置在可以轻松集成到智能眼镜框架中的位置。本文描述了详细的硬件设计以及对适当信号处理和机器学习方法的选择和调整。为了进行评估，我们要求八名参与者模仿一组面部动作，如表达快乐、愤怒、惊讶、悲伤、沮丧和厌恶，以及手势，如亲吻、眨眼、伸出舌头和吃药。已经对一个包含 2640 个事件的完整数据集进行了 66%的训练和 33%的测试率的评估。尽管我们遇到了志愿者表情高度变化的情况，但我们的方法在用户独立的场景下表现出召回率为 55%、精度为 56%和 F1 得分为 54%（9%的机会水平）。在基于用户的基础上，我们最差的结果 F1 得分为 60%，最好的结果 F1 得分为 89%。对于快乐、愤怒、亲吻、伸出舌头和中立（Null-class）等表情，召回率≥60%。
