Zhang Xiang, Wang Yiwen
Annu Int Conf IEEE Eng Med Biol Soc. 2020 Jul;2020:3086-3089. doi: 10.1109/EMBC44109.2020.9175985.
Brain-Machine Interface (BMI) provides a promising way to help disabled people restore their motor functions. The patients are able to control the external devices directly from their neural signals by the decoder. Due to various reasons such as mental fatigue and distraction, the distribution of the neural signals might change, which might lead to poor performance for the decoder. In this case, we need to calibrate the parameters before each session, which needs the professionals to label the data and is not convenient for the patient's usage at home. In this paper, we propose a covariant cluster transfer mechanism for the kernel reinforcement learning (RL) algorithm to speed up the adaptation across sessions. The parameters of the decoder will adaptively change according to a reward signal, which could be easily set by the patient. More importantly, we cluster the neural patterns in previous sessions. The cluster represents the conditional distribution from neural patterns to actions. When a distinct neural pattern appears in the new session, the nearest cluster will be transferred. In this way, the knowledge from the old session could be utilized to accelerate the learning in the new session. Our proposed algorithm is tested on the simulated neural data where the neural signal's distribution differs across sessions. Compared with the training from random initialization and a weight transfer policy, our proposed cluster transfer mechanism maintains a significantly higher success rate and a faster adaptation when the conditional distribution from neural signals to actions remains similar.
脑机接口（BMI）为帮助残疾人恢复运动功能提供了一种很有前景的方法。患者能够通过解码器直接根据其神经信号控制外部设备。由于精神疲劳和注意力分散等各种原因，神经信号的分布可能会发生变化，这可能导致解码器性能不佳。在这种情况下，我们需要在每次会话前校准参数，这需要专业人员对数据进行标注，对患者在家使用来说不太方便。在本文中，我们为内核强化学习（RL）算法提出了一种协变聚类转移机制，以加速跨会话的适应。解码器的参数将根据奖励信号自适应变化，患者可以轻松设置该奖励信号。更重要的是，我们对先前会话中的神经模式进行聚类。该聚类代表从神经模式到动作的条件分布。当新会话中出现不同的神经模式时，最近的聚类将被转移。通过这种方式，可以利用旧会话中的知识来加速新会话中的学习。我们提出的算法在模拟神经数据上进行了测试，其中神经信号的分布在不同会话中有所不同。与从随机初始化和权重转移策略进行的训练相比，当从神经信号到动作的条件分布保持相似时，我们提出的聚类转移机制保持了显著更高的成功率和更快的适应性。
