Cao Guohua, Bai Jimeng
School of Mechanical and Electrical Engineering, Changchun University of Science and Technology, Changchun, China.
PLoS One. 2025 Feb 18;20(2):e0311550. doi: 10.1371/journal.pone.0311550. eCollection 2025.
Due to the complexity and variability of application scenarios and the increasing demands for assembly, single-agent algorithms often face challenges in convergence and exhibit poor performance in robotic arm assembly processes. To address these issues, this paper proposes a method that employs a multi-agent reinforcement learning algorithm for the shaft-hole assembly of robotic arms, with a specific focus on square shaft-hole assemblies. First, we analyze the stages of hole-seeking, alignment, and insertion in the shaft-hole assembly process, based on a comprehensive study of the interactions between shafts and holes. Next, a reward function is designed by integrating the decoupled multi-agent deterministic deep deterministic policy gradient (DMDDPG) algorithm. Finally, a simulation environment is created in Gazebo, using circular and square shaft-holes as experimental subjects to model the robotic arm's shaft-hole assembly. The simulation results indicate that the proposed algorithm, which models the first three joints and the last three joints of the robotic arm as multi-agents, demonstrates not only enhanced adaptability but also faster and more stable convergence.
由于应用场景的复杂性和多变性以及对装配要求的不断提高，单智能体算法在收敛方面常常面临挑战，并且在机器人手臂装配过程中表现出较差的性能。为了解决这些问题，本文提出了一种将多智能体强化学习算法应用于机器人手臂轴孔装配的方法，特别关注方轴孔装配。首先，在对轴与孔之间的相互作用进行全面研究的基础上，我们分析了轴孔装配过程中的找孔、对齐和插入阶段。接下来，通过集成解耦的多智能体确定性深度确定性策略梯度（DMDDPG）算法设计了一个奖励函数。最后，在Gazebo中创建了一个仿真环境，以圆形和方形轴孔作为实验对象对机器人手臂的轴孔装配进行建模。仿真结果表明，将机器人手臂的前三个关节和后三个关节建模为多智能体的所提出算法，不仅展示出增强的适应性，而且收敛更快、更稳定。
