Dong Yihang, Jing Changhong, Mahmud Mufti, Ng Michael Kwok-Po, Wang Shuqiang
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
University of Chinese Academy of Sciences, Chinese Academy of Sciences, Beijing, China.
Brain Inform. 2024 Dec 18;11(1):31. doi: 10.1186/s40708-024-00245-8.
Affective computing is a key research area in computer science, neuroscience, and psychology, aimed at enabling computers to recognize, understand, and respond to human emotional states. As the demand for affective computing technology grows, emotion recognition methods based on physiological signals have become research hotspots. Among these, electroencephalogram (EEG) signals, which reflect brain activity, are highly promising. However, due to individual physiological and anatomical differences, EEG signals introduce noise, reducing emotion recognition performance. Additionally, the synchronous collection of multimodal data in practical applications requires high equipment and environmental standards, limiting the practical use of EEG signals. To address these issues, this study proposes the Emotion Preceptor, a cross-subject emotion recognition model based on unimodal EEG signals. This model introduces a Static Spatial Adapter to integrate spatial information in EEG signals, reducing individual differences and extracting robust encoding information. The Temporal Causal Network then leverages temporal information to extract beneficial features for emotion recognition, achieving precise recognition based on unimodal EEG signals. Extensive experiments on the SEED and SEED-V datasets demonstrate the superior performance of the Emotion Preceptor and validate the effectiveness of the new data processing method that combines DE features in a temporal sequence. Additionally, we analyzed the model's data flow and encoding methods from a biological interpretability perspective and validated it with neuroscience research related to emotion generation and regulation, promoting further development in emotion recognition research based on EEG signals.
情感计算是计算机科学、神经科学和心理学中的一个关键研究领域，旨在使计算机能够识别、理解和响应人类的情绪状态。随着对情感计算技术需求的增长，基于生理信号的情感识别方法已成为研究热点。其中，反映大脑活动的脑电图（EEG）信号极具潜力。然而，由于个体生理和解剖差异，EEG信号会引入噪声，降低情感识别性能。此外，实际应用中多模态数据的同步采集需要高设备和环境标准，限制了EEG信号的实际应用。为解决这些问题，本研究提出了Emotion Preceptor，一种基于单模态EEG信号的跨主体情感识别模型。该模型引入了静态空间适配器来整合EEG信号中的空间信息，减少个体差异并提取鲁棒的编码信息。然后，时间因果网络利用时间信息提取对情感识别有益的特征，实现基于单模态EEG信号的精确识别。在SEED和SEED-V数据集上进行的大量实验证明了Emotion Preceptor的卓越性能，并验证了将DE特征按时间序列组合的新数据处理方法的有效性。此外，我们从生物可解释性的角度分析了模型的数据流和编码方法，并用与情感产生和调节相关的神经科学研究进行了验证，推动了基于EEG信号的情感识别研究的进一步发展。
