UNIDEMI, Department of Mechanical and Industrial Engineering, NOVA School of Science and Technology, Universidade NOVA de Lisboa, 2829-516 Caparica, Portugal.
Laboratório Associado de Sistemas Inteligentes, LASI, 4800-058 Guimarães, Portugal.
Sensors (Basel). 2023 Jan 3;23(1):553. doi: 10.3390/s23010553.
The human-robot collaboration (HRC) solutions presented so far have the disadvantage that the interaction between humans and robots is based on the human's state or on specific gestures purposely performed by the human, thus increasing the time required to perform a task and slowing down the pace of human labor, making such solutions uninteresting. In this study, a different concept of the HRC system is introduced, consisting of an HRC framework for managing assembly processes that are executed simultaneously or individually by humans and robots. This HRC framework based on deep learning models uses only one type of data, RGB camera data, to make predictions about the collaborative workspace and human action, and consequently manage the assembly process. To validate the HRC framework, an industrial HRC demonstrator was built to assemble a mechanical component. Four different HRC frameworks were created based on the convolutional neural network (CNN) model structures: Faster R-CNN ResNet-50 and ResNet-101, YOLOv2 and YOLOv3. The HRC framework with YOLOv3 structure showed the best performance, showing a mean average performance of 72.26% and allowed the HRC industrial demonstrator to successfully complete all assembly tasks within a desired time window. The HRC framework has proven effective for industrial assembly applications.
目前提出的人机协作 (HRC) 解决方案存在一个缺点，即人类和机器人之间的交互基于人类的状态或人类故意执行的特定手势，从而增加了执行任务所需的时间并降低了人类劳动的速度，使得此类解决方案变得无趣。在这项研究中，引入了一种不同的 HRC 系统概念，该系统由一个用于管理由人类和机器人同时或单独执行的装配过程的 HRC 框架组成。这种基于深度学习模型的 HRC 框架仅使用一种类型的数据（RGB 相机数据）来对协作工作空间和人类动作进行预测，并相应地管理装配过程。为了验证 HRC 框架，构建了一个工业 HRC 演示器来组装一个机械部件。根据卷积神经网络 (CNN) 模型结构创建了四个不同的 HRC 框架：Faster R-CNN ResNet-50 和 ResNet-101、YOLOv2 和 YOLOv3。具有 YOLOv3 结构的 HRC 框架表现出最佳性能，平均性能达到 72.26%，并使 HRC 工业演示器能够在所需的时间窗口内成功完成所有装配任务。HRC 框架已被证明可有效用于工业装配应用。
