Yolcu Gozde, Oztel Ismail, Kazan Serap, Oz Cemil, Palaniappan Kannappan, Lever Teresa E, Bunyak Filiz
Department of Computer Engineering, Sakarya University, 54050 Serdivan, Sakarya, Turkey.
Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO 65211, USA.
Multimed Tools Appl. 2019 Nov;78(22):31581-31603. doi: 10.1007/s11042-019-07959-6. Epub 2019 Jul 23.
Facial expressions are a significant part of non-verbal communication. Recognizing facial expressions of people with neurological disorders is essential because these people may have lost a significant amount of their verbal communication ability. Such an assessment requires time consuming examination involving medical personnel, which can be quite challenging and expensive. Automated facial expression recognition systems that are low-cost and noninvasive can help experts detect neurological disorders. In this study, an automated facial expression recognition system is developed using a novel deep learning approach. The architecture consists of four-stage networks. The first, second and third networks segment the facial components which are essential for facial expression recognition. Owing to the three networks, an iconize facial image is obtained. The fourth network classifies facial expressions using raw facial images and iconize facial images. This four-stage method combines holistic facial information with local part-based features to achieve more robust facial expression recognition. Preliminary experimental results achieved 94.44% accuracy for facial expression recognition on RaFD database. The proposed system produced 5% improvement than the facial expression recognition system by using raw images. This study presents a quantitative, objective and non-invasive facial expression recognition system to help in the monitoring and diagnosis of neurological disorders influencing facial expressions.
面部表情是非语言交流的重要组成部分。识别患有神经系统疾病的人的面部表情至关重要，因为这些人可能已经丧失了大量的语言交流能力。这样的评估需要医疗人员进行耗时的检查，这可能极具挑战性且成本高昂。低成本且无创的自动面部表情识别系统可以帮助专家检测神经系统疾病。在本研究中，使用一种新颖的深度学习方法开发了一种自动面部表情识别系统。该架构由四级网络组成。第一、第二和第三网络分割对面部表情识别至关重要的面部组件。借助这三个网络，获得了一个图标化的面部图像。第四网络使用原始面部图像和图标化面部图像对面部表情进行分类。这种四阶段方法将整体面部信息与基于局部部分的特征相结合，以实现更强大的面部表情识别。在RaFD数据库上面部表情识别的初步实验结果达到了94.44%的准确率。所提出的系统比使用原始图像的面部表情识别系统提高了5%。本研究提出了一种定量、客观且无创的面部表情识别系统，以帮助监测和诊断影响面部表情的神经系统疾病。
