Department of Biomedical Engineering, Hefei University of Technology, Hefei, 230009, China.
Department of Biomedical Engineering, Hefei University of Technology, Hefei, 230009, China; Anhui Province Key Laboratory of Measuring Theory and Precision Instrument, Hefei University of Technology, Hefei 230009, China.
Comput Biol Med. 2023 Apr;156:106689. doi: 10.1016/j.compbiomed.2023.106689. Epub 2023 Feb 24.
Emotion recognition is a key component of human-computer interaction technology, for which facial electromyogram (fEMG) is an important physiological modality. Recently, deep-learning-based emotion recognition using fEMG signals has drawn increased attention. However, the ability of effective feature extraction and the demand of large-scale training data are two dominant factors that restrict the performance of emotion recognition. In this paper, a novel spatio-temporal deep forest (STDF) model is proposed to classify three categories of discrete emotions (neutral, sadness, and fear) using multi-channel fEMG signals. The feature extraction module fully extracts effective spatio-temporal features of fEMG signals using a combination of 2D frame sequences and multi-grained scanning. Meanwhile, a cascade forest-based classifier is designed to provide optimal structures for different scales of training data via automatically adjusting the number of cascade layers. The proposed model and five comparison methods were evaluated on our in-house fEMG dataset that included three discrete emotions and three channels of fEMG electrodes with a total of twenty-seven subjects. Experimental results demonstrate that the proposed STDF model achieves the best recognition performance with an average accuracy of 97.41%. Besides, our proposed STDF model can reduced the scale of training data to 50% while the average accuracy of emotion recognition is only reduced by about 5%. Our proposed model offers an effective solution for practical applications of fEMG-based emotion recognition.
情感识别是人机交互技术的关键组成部分，其中面部肌电图（fEMG）是一种重要的生理模态。最近，基于深度学习的 fEMG 信号情感识别引起了越来越多的关注。然而，有效特征提取的能力和大规模训练数据的需求是限制情感识别性能的两个主要因素。在本文中，提出了一种新颖的时空深度森林（STDF）模型，用于使用多通道 fEMG 信号对三类离散情感（中性、悲伤和恐惧）进行分类。特征提取模块使用二维帧序列和多粒度扫描的组合，充分提取 fEMG 信号的有效时空特征。同时，设计了基于级联森林的分类器，通过自动调整级联层数，为不同规模的训练数据提供最优结构。在我们的内部 fEMG 数据集上评估了所提出的模型和五种比较方法，该数据集包括三种离散情感和三个 fEMG 电极通道，共有二十七个受试者。实验结果表明，所提出的 STDF 模型的识别性能最佳，平均准确率为 97.41%。此外，我们提出的 STDF 模型可以将训练数据的规模减少到 50%，而情感识别的平均准确率仅降低约 5%。我们提出的模型为基于 fEMG 的情感识别的实际应用提供了有效的解决方案。
