Qin Yuxin, Li Baojiang, Wang Wenlong, Shi Xingbin, Peng Cheng, Wang Xichao, Wang Haiyan
The School of Electrical Engineering, Shanghai DianJi University, Shanghai, People's Republic of China.
Intelligent Decision and Control Technology Institute, Shanghai Dianji University, Shanghai, People's Republic of China.
J Neural Eng. 2025 Feb 7;22(1). doi: 10.1088/1741-2552/adaf58.
. Among all BCI paradigms, motion imagery (MI) has gained favor among researchers because it allows users to control external devices by imagining movements rather than actually performing actions. This property holds important promise for clinical applications, especially in areas such as stroke rehabilitation. Electroencephalogram (EEG) signals and functional near-infrared spectroscopy (fNIRS) signals are two of the more popular neuroimaging techniques for obtaining MI signals from the brain. However, the performance of MI-based unimodal classification methods is low due to the limitations of EEG or fNIRS.. In this paper, we propose a new multimodal fusion classification method capable of combining the potential complementary advantages of EEG and fNIRS. First, we propose a feature extraction network capable of extracting spatio-temporal features from EEG-based and fNIRS-based MI signals. Then, we successively fused the EEG and fNIRS at the feature-level and the decision-level to improve the adaptability and robustness of the model.. We validate the performance of ECA-FusionNet on a publicly available EEG-fNIRS dataset. The results show that ECA-FusionNet outperforms unimodal classification methods, as well as existing fusion classification methods, in terms of classification accuracy for MI.. ECA-FusionNet may provide a useful reference for the field of multimodal fusion classification.
在所有脑机接口范式中，运动想象（MI）在研究人员中颇受青睐，因为它允许用户通过想象动作而非实际执行动作来控制外部设备。这一特性在临床应用中具有重要前景，尤其是在中风康复等领域。脑电图（EEG）信号和功能近红外光谱（fNIRS）信号是从大脑获取MI信号的两种较受欢迎的神经成像技术。然而，由于EEG或fNIRS的局限性，基于MI的单模态分类方法的性能较低。在本文中，我们提出了一种新的多模态融合分类方法，该方法能够结合EEG和fNIRS的潜在互补优势。首先，我们提出了一种能够从基于EEG和基于fNIRS的MI信号中提取时空特征的特征提取网络。然后，我们在特征级和决策级依次融合EEG和fNIRS，以提高模型的适应性和鲁棒性。我们在一个公开可用的EEG-fNIRS数据集上验证了ECA-FusionNet的性能。结果表明，在MI的分类准确率方面，ECA-FusionNet优于单模态分类方法以及现有的融合分类方法。ECA-FusionNet可能为多模态融合分类领域提供有用的参考。
