Department of Biomedical Engineering, University of Miami Coral Gables, FL, USA.
Department of Biomedical Engineering, University of Miami Coral Gables, FL, USA ; Department of Neuroscience, University of Miami Coral Gables, FL, USA ; Miami Project to Cure Paralysis, University of Miami Coral Gables, FL, USA.
Front Neurosci. 2014 May 26;8:111. doi: 10.3389/fnins.2014.00111. eCollection 2014.
Brain-Machine Interfaces (BMIs) can be used to restore function in people living with paralysis. Current BMIs require extensive calibration that increase the set-up times and external inputs for decoder training that may be difficult to produce in paralyzed individuals. Both these factors have presented challenges in transitioning the technology from research environments to activities of daily living (ADL). For BMIs to be seamlessly used in ADL, these issues should be handled with minimal external input thus reducing the need for a technician/caregiver to calibrate the system. Reinforcement Learning (RL) based BMIs are a good tool to be used when there is no external training signal and can provide an adaptive modality to train BMI decoders. However, RL based BMIs are sensitive to the feedback provided to adapt the BMI. In actor-critic BMIs, this feedback is provided by the critic and the overall system performance is limited by the critic accuracy. In this work, we developed an adaptive BMI that could handle inaccuracies in the critic feedback in an effort to produce more accurate RL based BMIs. We developed a confidence measure, which indicated how appropriate the feedback is for updating the decoding parameters of the actor. The results show that with the new update formulation, the critic accuracy is no longer a limiting factor for the overall performance. We tested and validated the system onthree different data sets: synthetic data generated by an Izhikevich neural spiking model, synthetic data with a Gaussian noise distribution, and data collected from a non-human primate engaged in a reaching task. All results indicated that the system with the critic confidence built in always outperformed the system without the critic confidence. Results of this study suggest the potential application of the technique in developing an autonomous BMI that does not need an external signal for training or extensive calibration.
脑机接口（BMI）可用于恢复瘫痪患者的功能。当前的 BMI 需要进行广泛的校准，这增加了解码器训练的设置时间和外部输入，而瘫痪患者可能难以产生这些输入。这两个因素都给该技术从研究环境向日常生活活动（ADL）的过渡带来了挑战。为了使 BMI 在 ADL 中无缝使用，应该以最小的外部输入来处理这些问题，从而减少技术人员/护理人员校准系统的需求。基于强化学习（RL）的 BMI 是在没有外部训练信号时的一种很好的工具，并且可以为训练 BMI 解码器提供一种自适应模式。然而，基于 RL 的 BMI 对提供给适应 BMI 的反馈很敏感。在演员-评论家 BMI 中，该反馈由评论家提供，并且整个系统性能受到评论家准确性的限制。在这项工作中，我们开发了一种自适应 BMI，它可以处理评论家反馈中的不准确性，以产生更准确的基于 RL 的 BMI。我们开发了一种置信度度量，它指示反馈对于更新演员的解码参数有多合适。结果表明，使用新的更新公式，评论家的准确性不再是整体性能的限制因素。我们在三个不同的数据集上测试和验证了该系统：由 Izhikevich 神经尖峰模型生成的合成数据、具有高斯噪声分布的合成数据以及从参与伸手任务的非人类灵长类动物收集的数据。所有结果均表明，内置评论家置信度的系统始终优于没有评论家置信度的系统。这项研究的结果表明，该技术有可能开发出不需要外部信号进行训练或广泛校准的自主 BMI。
