Simistira Liwicki Foteini, Gupta Vibha, Saini Rajkumar, De Kanjar, Liwicki Marcus
Embedded Intelligent Systems LAB, Machine Learning, Department of Computer Science, Electrical and Space Engineering, Luleå University of Technology, 97187 Luleå, Sweden;
NeuroSci. 2022 Apr 19;3(2):226-244. doi: 10.3390/neurosci3020017. eCollection 2022 Jun.
This study focuses on the automatic decoding of inner speech using noninvasive methods, such as Electroencephalography (EEG). While inner speech has been a research topic in philosophy and psychology for half a century, recent attempts have been made to decode nonvoiced spoken words by using various brain-computer interfaces. The main shortcomings of existing work are reproducibility and the availability of data and code. In this work, we investigate various methods (using Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU), Long Short-Term Memory Networks (LSTM)) for the detection task of five vowels and six words on a publicly available EEG dataset. The main contributions of this work are (1) subject dependent vs. subject-independent approaches, (2) the effect of different preprocessing steps (Independent Component Analysis (ICA), down-sampling and filtering), and (3) word classification (where we achieve state-of-the-art performance on a publicly available dataset). Overall we achieve a performance accuracy of 35.20% and 29.21% when classifying five vowels and six words, respectively, in a publicly available dataset, using our tuned iSpeech-CNN architecture. All of our code and processed data are publicly available to ensure reproducibility. As such, this work contributes to a deeper understanding and reproducibility of experiments in the area of inner speech detection.
本研究聚焦于使用非侵入性方法（如脑电图（EEG））对内心言语进行自动解码。虽然内心言语在哲学和心理学领域已成为一个研究课题达半个世纪之久，但最近已有尝试通过使用各种脑机接口来解码无声说出的单词。现有工作的主要缺点在于可重复性以及数据和代码的可用性。在这项工作中，我们在一个公开可用的EEG数据集上，研究了多种方法（使用卷积神经网络（CNN）、门控循环单元（GRU）、长短期记忆网络（LSTM））用于五个元音和六个单词的检测任务。这项工作的主要贡献在于：（1）依赖个体与独立于个体的方法；（2）不同预处理步骤（独立成分分析（ICA）、下采样和滤波）的效果；（3）单词分类（我们在一个公开可用数据集上达到了当前最优性能）。总体而言，使用我们调优后的iSpeech - CNN架构，在一个公开可用数据集中对五个元音和六个单词进行分类时，我们分别实现了35.20%和29.21%的性能准确率。我们所有的代码和处理后的数据都公开可用，以确保可重复性。因此，这项工作有助于更深入地理解内心言语检测领域的实验并实现其可重复性。
