Wang Fei, Ren Baiming, Liu Yue, Cui Ben
Faculty of Robot Science and Engineering, Northeastern University, NO. 195, Innovation Road, Hunnan District, Shenyang, People's Republic of China.
College of Information Science and Engineering, Northeastern University, NO. 3-11, Wenhua Road, Heping District, Shenyang, People's Republic of China.
Rev Sci Instrum. 2022 Apr 1;93(4):045108. doi: 10.1063/5.0087561.
In this paper, an image-based visual servoing (IBVS) controller with a 6 degree-of-freedom robotic manipulator that tracks moving objects is investigated using the proposed Deep Q-Networks and proportional-integral-derivative (DQN-PID) controller. First, the classical IBVS controller and the problem of feature loss and large steady-state error for tracking moving targets are introduced. Then, a DQN-PID based IBVS method is proposed to solve the problem of feature loss and large steady-state error and improve the servo precision, as the existing methods are hard to use for solve the problems. Specifically, the IBVS method is inherited by our controller to build the tracking model, and a value-based reinforcement learning method is proposed as an adaptive law for dynamically tuning the PID parameters in the discrete space, which can track the moving target and keep the servo feature in the field of the camera. Finally, compared with the different existing methods, the DQN-PID based IBVS method has merits of higher accuracy and more stable tracking, or generalization.
本文利用所提出的深度Q网络与比例积分微分（DQN-PID）控制器，对具有6自由度机器人操纵器的基于图像的视觉伺服（IBVS）控制器进行了研究，该控制器用于跟踪移动目标。首先，介绍了经典的IBVS控制器以及跟踪移动目标时的特征丢失和稳态误差大的问题。然后，提出了一种基于DQN-PID的IBVS方法来解决特征丢失和稳态误差大的问题，并提高伺服精度，因为现有方法难以解决这些问题。具体而言，我们的控制器继承了IBVS方法来建立跟踪模型，并提出了一种基于值的强化学习方法作为在离散空间中动态调整PID参数的自适应律，该方法可以跟踪移动目标并在相机视野内保持伺服特征。最后，与不同的现有方法相比，基于DQN-PID的IBVS方法具有更高的精度、更稳定的跟踪性能或泛化能力。
