School of Artificial Intelligence, South China Normal University, Guangzhou, 510630, China; Research Station in Mathematics, South China Normal University, Guangzhou, 510630, China.
School of Artificial Intelligence, South China Normal University, Guangzhou, 510630, China.
Neural Netw. 2024 Dec;180:106643. doi: 10.1016/j.neunet.2024.106643. Epub 2024 Aug 22.
Emotional recognition is highly important in the field of brain-computer interfaces (BCIs). However, due to the individual variability in electroencephalogram (EEG) signals and the challenges in obtaining accurate emotional labels, traditional methods have shown poor performance in cross-subject emotion recognition. In this study, we propose a cross-subject EEG emotion recognition method based on a semi-supervised fine-tuning self-supervised graph attention network (SFT-SGAT). First, we model multi-channel EEG signals by constructing a graph structure that dynamically captures the spatiotemporal topological features of EEG signals. Second, we employ a self-supervised graph attention neural network to facilitate model training, mitigating the impact of signal noise on the model. Finally, a semi-supervised approach is used to fine-tune the model, enhancing its generalization ability in cross-subject classification. By combining supervised and unsupervised learning techniques, the SFT-SGAT maximizes the utility of limited labeled data in EEG emotion recognition tasks, thereby enhancing the model's performance. Experiments based on leave-one-subject-out cross-validation demonstrate that SFT-SGAT achieves state-of-the-art cross-subject emotion recognition performance on the SEED and SEED-IV datasets, with accuracies of 92.04% and 82.76%, respectively. Furthermore, experiments conducted on a self-collected dataset comprising ten healthy subjects and eight patients with disorders of consciousness (DOCs) revealed that the SFT-SGAT attains high classification performance in healthy subjects (maximum accuracy of 95.84%) and was successfully applied to DOC patients, with four patients achieving emotion recognition accuracies exceeding 60%. The experiments demonstrate the effectiveness of the proposed SFT-SGAT model in cross-subject EEG emotion recognition and its potential for assessing levels of consciousness in patients with DOC.
情绪识别在脑机接口（BCI）领域中至关重要。然而，由于脑电图（EEG）信号的个体可变性以及获取准确情绪标签的挑战，传统方法在跨被试情绪识别方面表现不佳。在这项研究中，我们提出了一种基于半监督微调自监督图注意网络（SFT-SGAT）的跨被试 EEG 情绪识别方法。首先，我们通过构建一个动态捕获 EEG 信号时空拓扑特征的图结构来对多通道 EEG 信号进行建模。其次，我们采用自监督图注意神经网络来促进模型训练，减轻信号噪声对模型的影响。最后，采用半监督方法对模型进行微调，增强其在跨被试分类中的泛化能力。通过结合监督和无监督学习技术，SFT-SGAT 最大限度地利用 EEG 情绪识别任务中有限的标记数据的效用，从而提高模型的性能。基于留一被试交叉验证的实验表明，SFT-SGAT 在 SEED 和 SEED-IV 数据集上实现了最先进的跨被试情绪识别性能，准确率分别为 92.04%和 82.76%。此外，在由 10 名健康受试者和 8 名意识障碍（DOC）患者组成的自采集数据集上进行的实验表明，SFT-SGAT 在健康受试者中实现了高分类性能（最大准确率为 95.84%），并成功应用于 DOC 患者，其中 4 名患者的情绪识别准确率超过 60%。实验证明了所提出的 SFT-SGAT 模型在跨被试 EEG 情绪识别中的有效性及其在评估 DOC 患者意识水平方面的潜力。
