Li Guanda, Hayashibe Mitsuhiro
Neuro-Robotics Lab, Department of Robotics, Graduate School of Engineering, Tohoku University, Sendai, Japan.
Sci Rep. 2025 Jan 3;15(1):712. doi: 10.1038/s41598-024-82472-x.
Humans exploit motor synergies for motor control; however, how they emerge during motor learning is not clearly understood. Few studies have dealt with the computational mechanism for generating synergies. Previously, optimal control generated synergistic motion for the upper limb; however, it has not yet been applied to the high-dimensional whole-body system. We investigated the emergence of synergies through deep reinforcement learning of whole-body locomotion tasks. We carried out a joint-space synergy analysis on whole-body control solutions for walking and running agents in simulated environments. Although a synergy constraint was never encoded into the reward function, the synergy emerged during the learning of walking and running tasks. To investigate the effect of gait symmetry on synergy emergence, we varied the weight level of symmetry loss. Interestingly, increasing the weight of symmetry loss resulted in increased energy efficiency and synergetic motion patterns concurrently. These results illustrate the correlation between motor synergy, energy efficiency, and gait symmetry in whole-body motor learning, reflecting that deep reinforcement learning can generate synergistic gait for highly redundant joint systems, similar to human motor control. This suggests that locomotor synergies can emerge through learning processes, complementing the understanding of synergy emergence mechanisms.
人类利用运动协同作用进行运动控制；然而，它们在运动学习过程中是如何出现的，目前还不清楚。很少有研究涉及产生协同作用的计算机制。此前，最优控制为上肢生成了协同运动；然而，它尚未应用于高维全身系统。我们通过对全身运动任务进行深度强化学习来研究协同作用的出现。我们对模拟环境中行走和跑步智能体的全身控制解决方案进行了关节空间协同分析。尽管协同约束从未被编码到奖励函数中，但协同作用在行走和跑步任务的学习过程中出现了。为了研究步态对称性对协同作用出现的影响，我们改变了对称性损失的权重水平。有趣的是，增加对称性损失的权重会同时提高能量效率和协同运动模式。这些结果说明了全身运动学习中运动协同、能量效率和步态对称性之间的相关性，反映出深度强化学习可以为高度冗余的关节系统生成协同步态，类似于人类的运动控制。这表明运动协同作用可以通过学习过程出现，补充了对协同作用出现机制的理解。
