Université Grenoble Alpes, CEA, LETI, Clinatec, F-38000 Grenoble, France.
Université Paris-Saclay, CEA, List, F-91120 Palaiseau, France.
J Neural Eng. 2022 Mar 31;19(2). doi: 10.1088/1741-2552/ac5d69.
Motor brain-computer interfaces (BCIs) are a promising technology that may enable motor-impaired people to interact with their environment. BCIs would potentially compensate for arm and hand function loss, which is the top priority for individuals with tetraplegia. Designing real-time and accurate BCI is crucial to make such devices useful, safe, and easy to use by patients in a real-life environment. Electrocorticography (ECoG)-based BCIs emerge as a good compromise between invasiveness of the recording device and good spatial and temporal resolution of the recorded signal. However, most ECoG signal decoders used to predict continuous hand movements are linear models. These models have a limited representational capacity and may fail to capture the relationship between ECoG signal features and continuous hand movements. Deep learning (DL) models, which are state-of-the-art in many problems, could be a solution to better capture this relationship.In this study, we tested several DL-based architectures to predict imagined 3D continuous hand translation using time-frequency features extracted from ECoG signals. The dataset used in the analysis is a part of a long-term clinical trial (ClinicalTrials.gov identifier: NCT02550522) and was acquired during a closed-loop experiment with a tetraplegic subject. The proposed architectures include multilayer perceptron, convolutional neural networks (CNNs), and long short-term memory networks (LSTM). The accuracy of the DL-based and multilinear models was compared offline using cosine similarity.Our results show that CNN-based architectures outperform the current state-of-the-art multilinear model. The best architecture exploited the spatial correlation between neighboring electrodes with CNN and benefited from the sequential character of the desired hand trajectory by using LSTMs. Overall, DL increased the average cosine similarity, compared to the multilinear model, by up to 60%, from 0.189 to 0.302 and from 0.157 to 0.249 for the left and right hand, respectively.This study shows that DL-based models could increase the accuracy of BCI systems in the case of 3D hand translation prediction in a tetraplegic subject.
运动脑机接口 (BCI) 是一种很有前途的技术,它可以使运动障碍者与环境进行交互。BCI 可能会补偿手臂和手部功能的丧失,这是四肢瘫痪患者的首要任务。设计实时和准确的 BCI 对于使此类设备在现实环境中对患者有用、安全且易于使用至关重要。基于皮层电图 (ECoG) 的 BCI 在记录设备的侵入性和记录信号的良好空间和时间分辨率之间提供了很好的折衷。然而,用于预测连续手部运动的大多数 ECoG 信号解码器都是线性模型。这些模型的表示能力有限,可能无法捕捉 ECoG 信号特征与连续手部运动之间的关系。深度学习 (DL) 模型在许多问题中是最先进的,可能是更好地捕捉这种关系的解决方案。在这项研究中,我们测试了几种基于 DL 的架构,以使用从 ECoG 信号中提取的时频特征来预测想象中的 3D 连续手部平移。分析中使用的数据集是一项长期临床试验的一部分(ClinicalTrials.gov 标识符:NCT02550522),是在一名四肢瘫痪患者的闭环实验中获得的。所提出的架构包括多层感知器、卷积神经网络 (CNN) 和长短时记忆网络 (LSTM)。离线使用余弦相似度比较了基于 DL 和多线性模型的准确性。我们的结果表明,基于 CNN 的架构优于当前最先进的多线性模型。最佳架构利用了 CNN 中相邻电极之间的空间相关性,并通过使用 LSTM 受益于所需手部轨迹的顺序特征。总体而言,与多线性模型相比,DL 将平均余弦相似度提高了 60%,从 0.189 提高到 0.302,从 0.157 提高到 0.249,分别用于左手和右手。这项研究表明,在四肢瘫痪患者的 3D 手部平移预测情况下,基于 DL 的模型可以提高 BCI 系统的准确性。