Wang Fei, Wu Shichao, Zhang Weiwei, Xu Zongfeng, Zhang Yahui, Wu Chengdong, Coleman Sonya
Faculty of Robot Science and Engineering, Northeastern University, Shenyang, 110169, China.
Faculty of Robot Science and Engineering, Northeastern University, Shenyang, 110169, China.
Neuropsychologia. 2020 Sep;146:107506. doi: 10.1016/j.neuropsychologia.2020.107506. Epub 2020 Jun 1.
Electroencephalogram (EEG), as a direct response to brain activity, can be used to detect mental states and physical conditions. Among various EEG-based emotion recognition studies, due to the non-linear, non-stationary and the individual difference of EEG signals, traditional recognition methods still have the disadvantages of complicated feature extraction and low recognition rates. Thus, this paper first proposes a novel concept of electrode-frequency distribution maps (EFDMs) with short-time Fourier transform (STFT). Residual block based deep convolutional neural network (CNN) is proposed for automatic feature extraction and emotion classification with EFDMs. Aim at the shortcomings of the small amount of EEG samples and the challenge of differences in individual emotions, which makes it difficult to construct a universal model, this paper proposes a cross-datasets emotion recognition method of deep model transfer learning. Experiments carried out on two publicly available datasets. The proposed method achieved an average classification score of 90.59% based on a short length of EEG data on SEED, which is 4.51% higher than the baseline method. Then, the pre-trained model was applied to DEAP through deep model transfer learning with a few samples, resulted an average accuracy of 82.84%. Finally, this paper adopts the gradient weighted class activation mapping (Grad-CAM) to get a glimpse of what features the CNN has learned during training from EFDMs and concludes that the high frequency bands are more favorable for emotion recognition.
脑电图（EEG）作为大脑活动的直接反应，可用于检测精神状态和身体状况。在各种基于EEG的情绪识别研究中，由于EEG信号的非线性、非平稳性以及个体差异，传统的识别方法仍然存在特征提取复杂和识别率低的缺点。因此，本文首先提出了一种基于短时傅里叶变换（STFT）的电极-频率分布图（EFDM）的新概念。提出了基于残差块的深度卷积神经网络（CNN），用于对EFDM进行自动特征提取和情绪分类。针对EEG样本数量少以及个体情绪差异带来的挑战，这使得构建通用模型变得困难，本文提出了一种深度模型迁移学习的跨数据集情绪识别方法。在两个公开可用的数据集上进行了实验。所提出的方法基于SEED上短长度的EEG数据实现了90.59%的平均分类得分，比基线方法高4.51%。然后，通过少量样本的深度模型迁移学习将预训练模型应用于DEAP，平均准确率为82.84%。最后，本文采用梯度加权类激活映射（Grad-CAM）来了解CNN在训练过程中从EFDM中学到了哪些特征，并得出高频带更有利于情绪识别的结论。
