Faculty of Computer Science and Information Technology, West Pomeranian University of Technology in Szczecin, ul. Żołnierska 49, 71-210 Szczecin, Poland.
Sensors (Basel). 2022 Aug 29;22(17):6518. doi: 10.3390/s22176518.
The paper presents a simple, yet robust computer vision system for robot arm tracking with the use of RGB-D cameras. Tracking means to measure in real time the robot state given by three angles and with known restrictions about the robot geometry. The tracking system consists of two parts: image preprocessing and machine learning. In the machine learning part, we compare two approaches: fitting the robot pose to the point cloud and fitting the convolutional neural network model to the sparse 3D depth images. The advantage of the presented approach is direct use of the point cloud transformed to the sparse image in the network input and use of sparse convolutional and pooling layers (sparse CNN). The experiments confirm that the robot tracking is performed in real time and with an accuracy comparable to the accuracy of the depth sensor.
本文提出了一种简单而强大的机器人手臂跟踪计算机视觉系统，该系统使用 RGB-D 相机。跟踪是指实时测量机器人的状态，给出三个角度，并已知机器人几何形状的限制。跟踪系统由两部分组成：图像预处理和机器学习。在机器学习部分，我们比较了两种方法：将机器人姿态拟合到点云中，以及将卷积神经网络模型拟合到稀疏 3D 深度图像中。所提出方法的优点是直接在网络输入中使用转换为稀疏图像的点云，并使用稀疏卷积和池化层（稀疏 CNN）。实验证实，机器人跟踪是实时进行的，并且精度可与深度传感器的精度相媲美。
