Jin Jing, Chen Weijie, Xu Ren, Liang Wei, Wu Xiao, He Xinjie, Wang Xingyu, Cichocki Andrzej
IEEE J Biomed Health Inform. 2025 Jan;29(1):198-209. doi: 10.1109/JBHI.2024.3472097. Epub 2025 Jan 7.
Motor imagery, one of the main brain-computer interface (BCI) paradigms, has been extensively utilized in numerous BCI applications, such as the interaction between disabled people and external devices. Precise decoding, one of the most significant aspects of realizing efficient and stable interaction, has received a great deal of intensive research. However, the current decoding methods based on deep learning are still dominated by single-scale serial convolution, which leads to insufficient extraction of abundant information from motor imagery signals. To overcome such challenges, we propose a new end-to-end convolutional neural network based on multiscale spatial-temporal feature fusion (MSTFNet) for EEG classification of motor imagery. The architecture of MSTFNet consists of four distinct modules: feature enhancement module, multiscale temporal feature extraction module, spatial feature extraction module and feature fusion module, with the latter being further divided into the depthwise separable convolution block and efficient channel attention block. Moreover, we implement a straightforward yet potent data augmentation strategy to bolster the performance of MSTFNet significantly. To validate the performance of MSTFNet, we conduct cross-session experiments and leave-one-subject-out experiments. The cross-session experiment is conducted across two public datasets and one laboratory dataset. On the public datasets of BCI Competition IV 2a and BCI Competition IV 2b, MSTFNet achieves classification accuracies of 83.62% and 89.26%, respectively. On the laboratory dataset, MSTFNet achieves 86.68% classification accuracy. Besides, the leave-one-subject-out experiment is performed on the BCI Competition IV 2a dataset, and MSTFNet achieves 66.31% classification accuracy. These experimental results outperform several state-of-the-art methodologies, indicate the proposed MSTFNet's robust capability in decoding EEG signals associated with motor imagery.
运动想象是脑机接口（BCI）的主要范式之一，已被广泛应用于众多BCI应用中，例如残疾人与外部设备之间的交互。精确解码是实现高效稳定交互的最重要方面之一，已经受到了大量深入研究。然而，当前基于深度学习的解码方法仍然以单尺度串行卷积为主，这导致从运动想象信号中提取丰富信息不足。为了克服这些挑战，我们提出了一种基于多尺度时空特征融合（MSTFNet）的新型端到端卷积神经网络，用于运动想象的脑电图分类。MSTFNet的架构由四个不同模块组成：特征增强模块、多尺度时间特征提取模块、空间特征提取模块和特征融合模块，后者进一步分为深度可分离卷积块和高效通道注意力块。此外，我们实施了一种简单而有效的数据增强策略，以显著提高MSTFNet的性能。为了验证MSTFNet的性能，我们进行了跨会话实验和留一受试者实验。跨会话实验是在两个公共数据集和一个实验室数据集上进行的。在BCI竞赛IV 2a和BCI竞赛IV 2b的公共数据集上，MSTFNet分别实现了83.62%和89.26%的分类准确率。在实验室数据集上，MSTFNet实现了86.68%的分类准确率。此外，留一受试者实验是在BCI竞赛IV 2a数据集上进行的，MSTFNet实现了66.31%的分类准确率。这些实验结果优于几种最先进的方法，表明所提出的MSTFNet在解码与运动想象相关的脑电信号方面具有强大的能力。
