Bagherzadeh Sara, Maghooli Keivan, Shalbaf Ahmad, Maghsoudi Arash
Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran.
Department of Biomedical Engineering and Medical Physics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
Cogn Neurodyn. 2022 Oct;16(5):1087-1106. doi: 10.1007/s11571-021-09756-0. Epub 2022 Jan 9.
Convolutional Neural Networks (CNN) have recently made considerable advances in the field of biomedical signal processing. These methodologies can assist in emotion recognition for affective brain computer interface. In this paper, a novel emotion recognition system based on the effective connectivity and the fine-tuned CNNs from multichannel Electroencephalogram (EEG) signal is presented. After preprocessing EEG signals, the relationships among 32 channels of EEG in the form of effective brain connectivity analysis which represents information flow between regions are computed by direct Directed Transfer Function (dDTF) method which yields a 32*32 image. Then, these constructed images from EEG signals for each subject were fed as input to four versions of pre-trained CNN models, AlexNet, ResNet-50, Inception-v3 and VGG-19 and the parameters of these models are fine-tuned, independently. The proposed deep learning architectures automatically learn patterns in the constructed image of the EEG signals in frequency bands. The efficiency of the proposed approach is evaluated on MAHNOB-HCI and DEAP databases. The experiments for classifying five emotional states show that the ResNet-50 applied on dDTF images in alpha band achieves best results due to specific architecture which captures the brain connectivity, efficiently. The accuracy and F1-score values for MAHNOB-HCI were obtained 99.41, 99.42 and for DEAP databases, 98.17, and 98.23. Newly proposed model is capable of effectively analyzing the brain function using information flow from multichannel EEG signals using effective connectivity measure of dDTF and ResNet-50.
卷积神经网络（CNN）最近在生物医学信号处理领域取得了显著进展。这些方法可以辅助情感脑机接口的情感识别。本文提出了一种基于有效连通性和来自多通道脑电图（EEG）信号的微调CNN的新型情感识别系统。在对EEG信号进行预处理后，通过直接定向传递函数（dDTF）方法计算以有效脑连通性分析形式表示区域间信息流的32通道EEG之间的关系，该方法产生一个32×32的图像。然后，将每个受试者的这些由EEG信号构建的图像作为输入馈送到四个预训练的CNN模型版本，即AlexNet、ResNet-50、Inception-v3和VGG-19，并独立地对这些模型的参数进行微调。所提出的深度学习架构自动学习频段中EEG信号构建图像中的模式。在MAHNOB-HCI和DEAP数据库上评估了所提出方法的效率。对五种情绪状态进行分类的实验表明，由于其特定的架构能够有效地捕捉脑连通性，应用于α波段dDTF图像的ResNet-50取得了最佳结果。MAHNOB-HCI的准确率和F1分数值分别为99.41、99.42，DEAP数据库的分别为98.17和98.23。新提出的模型能够使用dDTF的有效连通性度量和ResNet-50，通过多通道EEG信号的信息流有效地分析脑功能。
