Fujiwara Yosuke, Ushiba Junichi
Graduate School of Science and Technology, Keio University, Yokohama, Japan.
Information Services International-Dentsu, Ltd., Tokyo, Japan.
Front Comput Neurosci. 2022 May 20;16:882290. doi: 10.3389/fncom.2022.882290. eCollection 2022.
Concomitant with the development of deep learning, brain-computer interface (BCI) decoding technology has been rapidly evolving. Convolutional neural networks (CNNs), which are generally used as electroencephalography (EEG) classification models, are often deployed in BCI prototypes to improve the estimation accuracy of a participant's brain activity. However, because most BCI models are trained, validated, and tested within-subject cross-validation and there is no corresponding generalization model, their applicability to unknown participants is not guaranteed. In this study, to facilitate the generalization of BCI model performance to unknown participants, we trained a model comprising multiple layers of residual CNNs and visualized the reasons for BCI classification to reveal the location and timing of neural activities that contribute to classification. Specifically, to develop a BCI that can distinguish between rest, left-hand movement, and right-hand movement tasks with high accuracy, we created multilayers of CNNs, inserted residual networks into the multilayers, and used a larger dataset than in previous studies. The constructed model was analyzed with gradient-class activation mapping (Grad-CAM). We evaluated the developed model subject cross-validation and found that it achieved significantly improved accuracy (85.69 ± 1.10%) compared with conventional models or without residual networks. Grad-CAM analysis of the classification of cases in which our model produced correct answers showed localized activity near the premotor cortex. These results confirm the effectiveness of inserting residual networks into CNNs for tuning BCI. Further, they suggest that recording EEG signals over the premotor cortex and some other areas contributes to high classification accuracy.
随着深度学习的发展，脑机接口（BCI）解码技术也在迅速发展。卷积神经网络（CNN）通常用作脑电图（EEG）分类模型，经常被部署在BCI原型中，以提高对参与者大脑活动的估计准确性。然而，由于大多数BCI模型是在受试者内交叉验证中进行训练、验证和测试的，并且没有相应的泛化模型，因此不能保证它们对未知参与者的适用性。在本研究中，为了促进BCI模型性能对未知参与者的泛化，我们训练了一个由多层残差CNN组成的模型，并将BCI分类的原因可视化，以揭示有助于分类的神经活动的位置和时间。具体来说，为了开发一种能够高精度区分休息、左手运动和右手运动任务的BCI，我们创建了多层CNN，在多层中插入残差网络，并使用了比以前研究更大的数据集。使用梯度类激活映射（Grad-CAM）对构建的模型进行分析。我们通过受试者交叉验证对开发的模型进行评估，发现与传统模型或没有残差网络的模型相比，它的准确率显著提高（85.69±1.10%）。对我们的模型给出正确答案的案例分类进行的Grad-CAM分析显示，在运动前皮层附近有局部活动。这些结果证实了在CNN中插入残差网络以调整BCI的有效性。此外，它们表明在运动前皮层和其他一些区域记录EEG信号有助于提高分类准确率。
