IEEE J Biomed Health Inform. 2023 Aug;27(8):3698-3709. doi: 10.1109/JBHI.2023.3260816. Epub 2023 Aug 7.
Many clinical studies have shown that facial expression recognition and cognitive function are impaired in depressed patients. Different from spontaneous facial expression mimicry (SFEM), 164 subjects (82 in a case group and 82 in a control group) participated in our voluntary facial expression mimicry (VFEM) experiment using expressions of neutrality, anger, disgust, fear, happiness, sadness and surprise. Our research is as follows. First, we collected a large amount of subject data for VFEM. Second, we extracted the geometric features of subject facial expression images for VFEM and used Spearman correlation analysis, a random forest, and logistic regression-based recursive feature elimination (LR-RFE) to perform feature selection. The features selected revealed the difference between the case group and the control group. Third, we combined geometric features with the original images and improved the advanced deep learning facial expression recognition (FER) algorithms in different systems. We propose the E-ViT and E-ResNet based on VFEM. The accuracies and F1 scores were higher than those of the baseline models, respectively. Our research proved that it is effective to use feature selection to screen geometric features and combine them with a deep learning model for depression facial expression recognition.
许多临床研究表明，抑郁症患者的面部表情识别和认知功能受损。与自发性面部表情模仿（SFEM）不同，164 名受试者（病例组 82 名，对照组 82 名）参与了我们使用中性、愤怒、厌恶、恐惧、快乐、悲伤和惊讶表情的自愿性面部表情模仿（VFEM）实验。我们的研究如下。首先，我们收集了大量的 VFEM 主体数据。其次，我们提取了 VFEM 主体面部表情图像的几何特征，并使用 Spearman 相关分析、随机森林和基于逻辑回归的递归特征消除（LR-RFE）进行特征选择。选择的特征揭示了病例组和对照组之间的差异。第三，我们将几何特征与原始图像相结合，并改进了不同系统中的先进深度学习面部表情识别（FER）算法。我们提出了基于 VFEM 的 E-ViT 和 E-ResNet。与基线模型相比，这两种模型的准确率和 F1 评分都更高。我们的研究证明，使用特征选择来筛选几何特征并将其与深度学习模型结合进行抑郁面部表情识别是有效的。
