Zhang Yan, Fütterer Richard, Notni Gunther
Group for Quality Assurance and Industrial Image Processing, Technische Universität Ilmenau, Ilmenau, Germany.
Fraunhofer Institute for Applied Optics and Precision Engineering IOF Jena, Jena, Germany.
Front Robot AI. 2023 Mar 16;10:1120357. doi: 10.3389/frobt.2023.1120357. eCollection 2023.
The concept of Industry 4.0 brings the change of industry manufacturing patterns that become more efficient and more flexible. In response to this tendency, an efficient robot teaching approach without complex programming has become a popular research direction. Therefore, we propose an interactive finger-touch based robot teaching schema using a multimodal 3D image (color (RGB), thermal (T) and point cloud (3D)) processing. Here, the resulting heat trace touching the object surface will be analyzed on multimodal data, in order to precisely identify the true hand/object contact points. These identified contact points are used to calculate the robot path directly. To optimize the identification of the contact points we propose a calculation scheme using a number of anchor points which are first predicted by hand/object point cloud segmentation. Subsequently a probability density function is defined to calculate the prior probability distribution of true finger trace. The temperature in the neighborhood of each anchor point is then dynamically analyzed to calculate the likelihood. Experiments show that the trajectories estimated by our multimodal method have significantly better accuracy and smoothness than only by analyzing point cloud and static temperature distribution.
工业4.0的概念带来了工业制造模式的变革，使其变得更高效、更灵活。为响应这一趋势，一种无需复杂编程的高效机器人示教方法已成为热门研究方向。因此，我们提出了一种基于交互式手指触摸的机器人示教模式，该模式采用多模态3D图像（彩色（RGB）、热成像（T）和点云（3D））处理。在此，将在多模态数据上分析接触物体表面产生的热迹，以便精确识别真实的手/物体接触点。这些识别出的接触点直接用于计算机器人路径。为优化接触点的识别，我们提出一种计算方案，该方案使用多个锚点，这些锚点首先通过手/物体点云分割进行预测。随后定义概率密度函数来计算真实手指轨迹的先验概率分布。然后动态分析每个锚点附近的温度以计算似然性。实验表明，与仅通过分析点云和静态温度分布相比，我们的多模态方法估计的轨迹具有显著更高的准确性和平滑性。
