IEEE Trans Neural Syst Rehabil Eng. 2021;29:1016-1025. doi: 10.1109/TNSRE.2021.3081056. Epub 2021 Jun 8.
High-level spinal cord injuries often result in paralysis of all four limbs, leading to decreased patient independence and quality of life. Coordinated functional electrical stimulation (FES) of paralyzed muscles can be used to restore some motor function in the upper extremity. To coordinate functional movements, FES controllers should be developed to exploit the complex characteristics of human movement and produce the intended movement kinematics and/or kinetics. Here, we demonstrate the ability of a controller trained using reinforcement learning to generate desired movements of a horizontal planar musculoskeletal model of the human arm with 2 degrees of freedom and 6 actuators. The controller is given information about the kinematics of the arm, but not the internal state of the actuators. In particular, we demonstrate that a technique called "hindsight experience replay" can improve controller performance while also decreasing controller training time.
高位脊髓损伤常导致四肢瘫痪，导致患者独立性和生活质量下降。瘫痪肌肉的协调功能性电刺激 (FES) 可用于恢复上肢的一些运动功能。为了协调功能运动，应开发 FES 控制器以利用人类运动的复杂特征并产生预期的运动运动学和/或动力学。在这里，我们展示了使用强化学习训练的控制器生成具有 2 个自由度和 6 个执行器的人体手臂水平平面肌肉骨骼模型的期望运动的能力。控制器提供有关手臂运动学的信息，但不提供执行器的内部状态。特别是，我们证明了一种称为“后见之明经验重放”的技术可以在提高控制器性能的同时减少控制器训练时间。
