Annu Int Conf IEEE Eng Med Biol Soc. 2021 Nov;2021:4583-4587. doi: 10.1109/EMBC46164.2021.9630547.
Quadruped system is an animal-like model which has long been analyzed in terms of energy efficiency during its various gait locomotion. The generation of certain gait modes on these systems has been achieved by classical controllers which demand highly specific domain-knowledge and empirical parameter tuning. In this paper, we propose to use deep reinforcement learning (DRL) as an alternative approach to generate certain gait modes on quadrupeds, allowing potentially the same energetic analysis without the difficulty of designing an ad hoc controller. We show that by specifying a gait mode in the process of learning, it allows faster convergence of the learning process while at the same time imposing a certain gait type on the systems as opposed to the case without any gait specification. We demonstrate the advantages of using DRL as it can exploit automatically the physical condition of the robots such as the passive spring effect between the joints during the learning process, similar to the adaptation skills of an animal. The proposed system would provide a framework for quadrupedal trot-gallop energetic analysis for different body structures, body mass distributions and joint characteristics using DRL.
四足系统是一种类似动物的模型，长期以来一直从能量效率的角度对其各种步态运动进行分析。这些系统上某些步态模式的生成是通过需要高度特定领域知识和经验参数调整的经典控制器来实现的。在本文中，我们提出使用深度强化学习（DRL）作为生成四足动物某些步态模式的替代方法，在不设计特定于控制器的情况下，允许进行潜在的相同能量分析。我们表明，通过在学习过程中指定步态模式，可以加快学习过程的收敛速度，同时相对于没有任何步态规范的情况，将某种步态类型强加给系统。我们展示了使用 DRL 的优势，因为它可以在学习过程中自动利用机器人的物理条件，例如关节之间的被动弹簧效应，类似于动物的适应技能。所提出的系统将为使用 DRL 对不同的身体结构、质量分布和关节特性进行四足动物小跑-疾驰的能量分析提供一个框架。
