Zhang Hongyan, Liang Huawei, Ni Tao, Huang Lingtao, Yang Jinsong
School of Mechanical and Aerospace Engineering, Jilin University, Changchun 130022, China.
School of Vehicle and Energy, Yanshan University, Qinhuangdao 066004, China.
Sensors (Basel). 2021 Sep 17;21(18):6238. doi: 10.3390/s21186238.
As a complex task, robot sorting has become a research hotspot. In order to enable robots to perform simple, efficient, stable and accurate sorting operations for stacked multi-objects in unstructured scenes, a robot multi-object sorting system is built in this paper. Firstly, the training model of rotating target detection is constructed, and the placement state of five common objects in unstructured scenes is collected as the training set for training. The trained model is used to obtain the position, rotation angle and category of the target object. Then, the instance segmentation model is constructed, and the same data set is made, and the instance segmentation network model is trained. Then, the optimized Mask R-CNN instance segmentation network is used to segment the object surface pixels, and the upper surface point cloud is extracted to calculate the normal vector. Then, the angle obtained by the normal vector of the upper surface and the rotation target detection network is fused with the normal vector to obtain the attitude of the object. At the same time, the grasping order is calculated according to the average depth of the surface. Finally, after the obtained object posture, category and grasping sequence are fused, the performance of the rotating target detection network, the instance segmentation network and the robot sorting system are tested on the established experimental platform. Based on this system, this paper carried out an experiment on the success rate of object capture in a single network and an integrated network. The experimental results show that the multi-object sorting system based on deep learning proposed in this paper can sort stacked objects efficiently, accurately and stably in unstructured scenes.
作为一项复杂任务，机器人分拣已成为研究热点。为使机器人能够在非结构化场景中对堆叠的多物体进行简单、高效、稳定且准确的分拣操作，本文构建了一个机器人多物体分拣系统。首先，构建旋转目标检测的训练模型，收集非结构化场景中五个常见物体的放置状态作为训练集进行训练。利用训练好的模型获取目标物体的位置、旋转角度和类别。然后，构建实例分割模型，制作相同的数据集，并训练实例分割网络模型。接着，使用优化后的Mask R-CNN实例分割网络对物体表面像素进行分割，提取上表面点云以计算法向量。再将上表面法向量与旋转目标检测网络得到的角度进行融合，从而获得物体姿态。同时，根据表面平均深度计算抓取顺序。最后，将得到的物体姿态、类别和抓取顺序进行融合后，在搭建的实验平台上对旋转目标检测网络、实例分割网络及机器人分拣系统的性能进行测试。基于该系统，本文针对单网络和集成网络中的物体抓取成功率进行了实验。实验结果表明，本文提出的基于深度学习的多物体分拣系统能够在非结构化场景中高效、准确且稳定地对堆叠物体进行分拣。
