个体严重的肢体和言语瘫痪中使用言语神经假体实现可泛化的拼写

Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis.

机构信息

Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, USA.

Weill Institute for Neuroscience, University of California, San Francisco, San Francisco, CA, USA.

出版信息

Nat Commun. 2022 Nov 8;13(1):6510. doi: 10.1038/s41467-022-33611-3.

Abstract

Neuroprostheses have the potential to restore communication to people who cannot speak or type due to paralysis. However, it is unclear if silent attempts to speak can be used to control a communication neuroprosthesis. Here, we translated direct cortical signals in a clinical-trial participant (ClinicalTrials.gov; NCT03698149) with severe limb and vocal-tract paralysis into single letters to spell out full sentences in real time. We used deep-learning and language-modeling techniques to decode letter sequences as the participant attempted to silently spell using code words that represented the 26 English letters (e.g. "alpha" for "a"). We leveraged broad electrode coverage beyond speech-motor cortex to include supplemental control signals from hand cortex and complementary information from low- and high-frequency signal components to improve decoding accuracy. We decoded sentences using words from a 1,152-word vocabulary at a median character error rate of 6.13% and speed of 29.4 characters per minute. In offline simulations, we showed that our approach generalized to large vocabularies containing over 9,000 words (median character error rate of 8.23%). These results illustrate the clinical viability of a silently controlled speech neuroprosthesis to generate sentences from a large vocabulary through a spelling-based approach, complementing previous demonstrations of direct full-word decoding.

摘要

神经假体有可能恢复因瘫痪而无法说话或打字的人的交流能力。然而,目前尚不清楚是否可以利用无声的说话尝试来控制通信神经假体。在这里,我们将一名严重四肢和声带瘫痪的临床试验参与者(ClinicalTrials.gov;NCT03698149)的皮质直接信号转化为单个字母,实时拼写出完整的句子。我们使用深度学习和语言模型技术,将字母序列解码为参与者试图通过代表 26 个英文字母的代码词(例如“alpha”代表“a”)进行无声拼写。我们利用广泛的电极覆盖范围,超越了言语运动皮层,包括来自手部皮层的补充控制信号和来自低频和高频信号成分的补充信息,以提高解码准确性。我们使用词汇量为 1152 个单词的单词来解码句子,字符错误率中位数为 6.13%,速度为每分钟 29.4 个字符。在离线模拟中,我们表明我们的方法可以推广到大词汇量(包含超过 9000 个单词),字符错误率中位数为 8.23%。这些结果说明了通过拼写方法从大词汇量生成句子的无声控制语音神经假体的临床可行性,补充了先前直接全字解码的演示。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/f3f4/9643551/c71c7a387547/41467_2022_33611_Fig1_HTML.jpg

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索