Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China.
Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China.
J Neural Eng. 2021 Aug 31;18(4). doi: 10.1088/1741-2552/ac1ed2.
. Recently, transfer learning (TL) and deep learning (DL) have been introduced to solve intra- and inter-subject variability problems in brain-computer interfaces (BCIs). However, current TL and DL algorithms are usually validated within a single dataset, assuming that data of the test subjects are acquired under the same condition as that of training (source) subjects. This assumption is generally violated in practice because of different acquisition systems and experimental settings across studies and datasets. Thus, the generalization ability of these algorithms needs further validations in a cross-dataset scenario, which is closer to the actual situation. This study compared the transfer performance of pre-trained deep-learning models with different preprocessing strategies in a cross-dataset scenario.. This study used four publicly available motor imagery datasets, each was successively selected as a source dataset, and the others were used as target datasets. EEGNet and ShallowConvNet with four preprocessing strategies, namely channel normalization, trial normalization, Euclidean alignment, and Riemannian alignment, were trained with the source dataset. The transfer performance of pre-trained models was validated on the target datasets. This study also used adaptive batch normalization (AdaBN) for reducing interval covariate shift across datasets. This study compared the transfer performance of using the four preprocessing strategies and that of a baseline approach based on manifold embedded knowledge transfer (MEKT). This study also explored the possibility and performance of fusing MEKT and EEGNet.. The results show that DL models with alignment strategies had significantly better transfer performance than the other two preprocessing strategies. As an unsupervised domain adaptation method, AdaBN could also significantly improve the transfer performance of DL models. The transfer performance of DL models that combined AdaBN and alignment strategies significantly outperformed MEKT. Moreover, the generalizability of EEGNet models that combined AdaBN and alignment strategies could be further improved via the domain adaptation step in MEKT, achieving the best generalization ability among multiple datasets (BNCI2014001: 0.788, PhysionetMI: 0.679, Weibo2014: 0.753, Cho2017: 0.650).. The combination of alignment strategies and AdaBN could easily improve the generalizability of DL models without fine-tuning. This study may provide new insights into the design of transfer neural networks for BCIs by separating source and target batch normalization layers in the domain adaptation process.
. 最近，迁移学习（TL）和深度学习（DL）已被引入到脑机接口（BCI）中，以解决内在和主体间的可变性问题。然而，目前的 TL 和 DL 算法通常在单个数据集内进行验证，假设测试对象的数据是在与训练（源）对象相同的条件下获得的。在实践中，由于不同的研究和数据集的采集系统和实验设置不同，这一假设通常是不成立的。因此，这些算法的泛化能力需要在更接近实际情况的跨数据集场景中进一步验证。本研究在跨数据集场景中比较了具有不同预处理策略的预训练深度学习模型的转移性能。. 本研究使用了四个公开的运动想象数据集，每个数据集依次被选为源数据集，其他数据集被用作目标数据集。EEGNet 和 ShallowConvNet 采用了四种预处理策略，即通道归一化、试验归一化、欧几里得对齐和黎曼对齐，在源数据集上进行训练。然后在目标数据集上验证预训练模型的转移性能。本研究还使用自适应批量归一化（AdaBN）来减少跨数据集的区间协变量偏移。本研究比较了使用四种预处理策略的转移性能，以及基于流形嵌入知识转移（MEKT）的基线方法的转移性能。本研究还探索了融合 MEKT 和 EEGNet 的可能性和性能。. 结果表明，具有对齐策略的 DL 模型的转移性能明显优于其他两种预处理策略。作为一种无监督的域自适应方法，AdaBN 也可以显著提高 DL 模型的转移性能。结合 AdaBN 和对齐策略的 DL 模型的转移性能明显优于 MEKT。此外，通过 MEKT 中的域自适应步骤，可以进一步提高结合 AdaBN 和对齐策略的 EEGNet 模型的泛化能力，在多个数据集（BNCI2014001：0.788、PhysionetMI：0.679、Weibo2014：0.753、Cho2017：0.650）中实现了最好的泛化能力。. 对齐策略和 AdaBN 的结合可以在不进行微调的情况下轻松提高 DL 模型的可泛化性。本研究通过在域自适应过程中分离源和目标批量归一化层，为 BCI 中迁移神经网络的设计提供了新的思路。
