University of Toronto, Vector Institute for Artificial Intelligence; Toronto, Canada.
University of Toronto, Li Ka Shing Knowledge Institute, St Michael's Hospital; Toronto Canada.
J Neural Eng. 2020 Oct 13;17(5):056008. doi: 10.1088/1741-2552/abb7a7.
Most deep neural networks (DNNs) used as brain computer interfaces (BCI) classifiers are rarely viable for more than one person and are relatively shallow compared to the state-of-the-art in the wider machine learning literature. The goal of this work is to frame these as a unified challenge and reconsider how transfer learning is used to overcome these difficulties.
We present two variations of a holistic approach to transfer learning with DNNs for BCI that rely on a deeper network called TIDNet. Our approaches use multiple subjects for training in the interest of creating a more universal classifier that is applicable for new (unseen) subjects. The first approach is purely subject-invariant and the second targets specific subjects, without loss of generality. We use five publicly accessible datasets covering a range of tasks and compare our approaches to state-of-the-art alternatives in detail.
We observe that TIDNet in conjunction with our training augmentations is more consistent when compared to shallower baselines, and in some cases exhibits large and significant improvements, for instance motor imagery classification improvements of over 8%. Furthermore, we show that our suggested multi-domain learning (MDL) strategy strongly outperforms simply fine-tuned general models when targeting specific subjects, while remaining more generalizable to still unseen subjects.
TIDNet in combination with a data alignment-based training augmentation proves to be a consistent classification approach of single raw trials and can be trained even with the inclusion of corrupted trials. Our MDL strategy calls into question the intuition to fine-tune trained classifiers to new subjects, as it proves simpler and more accurate while remaining general. Furthermore, we show evidence that augmented TIDNet training makes better use of additional subjects, showing continued and greater performance improvement over shallower alternatives, indicating promise for a new subject-invariant paradigm rather than a subject-specific one.
大多数用于脑机接口（BCI）分类器的深度神经网络（DNN）很少适用于超过一个人，并且与更广泛的机器学习文献中的最新技术相比相对较浅。这项工作的目标是将这些问题作为一个统一的挑战，并重新考虑如何使用迁移学习来克服这些困难。
我们提出了两种用于 BCI 的基于 DNN 的整体迁移学习方法，这两种方法都依赖于一个称为 TIDNet 的更深的网络。我们的方法使用多个主题进行训练，以创建一个更通用的分类器，适用于新的（未见过的）主题。第一种方法是纯粹的主题不变，第二种方法是针对特定主题的，不失一般性。我们使用五个公开可用的数据集，涵盖了一系列任务，并详细比较了我们的方法与最新技术的替代方法。
我们观察到，与较浅的基线相比，TIDNet 结合我们的训练增强更具一致性，在某些情况下表现出较大且显著的改进，例如运动想象分类的改进超过 8%。此外，我们表明，我们提出的多域学习（MDL）策略在针对特定主题时，明显优于简单地微调通用模型，同时仍然更具通用性，适用于仍未见过的主题。
TIDNet 与基于数据对齐的训练增强相结合，被证明是一种单一原始试验的一致分类方法，甚至可以在包含损坏试验的情况下进行训练。我们的 MDL 策略对针对新主题微调训练好的分类器的直觉提出了质疑，因为它证明了更简单、更准确，同时仍然具有通用性。此外，我们还证明了增强 TIDNet 训练可以更好地利用额外的主题，与较浅的替代方案相比，持续且更大的性能提升，这表明在新的主题不变范例而不是主题特定范例方面有希望。
