IEEE Trans Neural Syst Rehabil Eng. 2019 Sep;27(9):1684-1694. doi: 10.1109/TNSRE.2019.2934176. Epub 2019 Aug 9.
Neuroprosthesis enables the brain control on the external devices purely using neural activity for paralyzed people. Supervised learning decoders recalibrate or re-fit the discrepancy between the desired target and decoder's output, where the correction may over-dominate the user's intention. Reinforcement learning decoder allows users to actively adjust their brain patterns through trial and error, which better represents the subject's motive. The computational challenge is to quickly establish new state-action mapping before the subject becomes frustrated. Recently proposed quantized attention-gated kernel reinforcement learning (QAGKRL) explores the optimal nonlinear neural-action mapping in the Reproducing Kernel Hilbert Space (RKHS). However, considering all past data in RKHS is less efficient and sensitive to detect the new neural patterns emerging in brain control. In this paper, we propose a clustering-based kernel RL algorithm. New neural patterns emerge and are clustered to represent the novel knowledge in brain control. The current neural data only activate the nearest subspace in RKHS for more efficient decoding. The dynamic clustering makes our algorithm more sensitive to new brain patterns. We test our algorithm on both the synthetic and real-world spike data. Compared with QAGKRL, our algorithm can achieve a quicker knowledge adaptation in brain control with less computational complexity.
神经假体仅使用神经活动就能使瘫痪患者对外部设备进行大脑控制。监督学习解码器重新校准或适配期望目标和解码器输出之间的差异，其中纠正可能会过度主导用户的意图。强化学习解码器允许用户通过反复试验主动调整大脑模式，这更好地代表了受试者的动机。计算上的挑战是在受试者感到沮丧之前快速建立新的状态-动作映射。最近提出的量化注意力门控核强化学习 (QAGKRL) 在再生核希尔伯特空间 (RKHS) 中探索了最优的非线性神经-动作映射。然而，考虑到 RKHS 中的所有过去数据效率较低，并且对检测大脑控制中出现的新神经模式不敏感。在本文中，我们提出了一种基于聚类的核 RL 算法。新的神经模式出现并聚类，以代表大脑控制中的新知识。当前的神经数据仅在 RKHS 中激活最近的子空间，以实现更高效的解码。动态聚类使我们的算法对新的大脑模式更加敏感。我们在合成和真实世界的尖峰数据上测试了我们的算法。与 QAGKRL 相比，我们的算法可以在大脑控制中实现更快的知识自适应，并且计算复杂度更低。
