Wunsch Lennard, Tenorio Christian Görner, Anding Katharina, Golomoz Andrei, Notni Gunther
Group of Quality Assurance and Industrial Image Processing, Faculty of Mechanical Engineering, Technische Universität Ilmenau, 98693 Ilmenau, Germany.
Fraunhofer Institute for Applied Optics and Precision Engineering IOF Jena, 07745 Jena, Germany.
J Imaging. 2024 Mar 21;10(3):73. doi: 10.3390/jimaging10030073.
Since 3D sensors became popular, imaged depth data are easier to obtain in the consumer sector. In applications such as defect localization on industrial objects or mass/volume estimation, precise depth data is important and, thus, benefits from the usage of multiple information sources. However, a combination of RGB images and depth images can not only improve our understanding of objects, capacitating one to gain more information about objects but also enhance data quality. Combining different camera systems using data fusion can enable higher quality data since disadvantages can be compensated. Data fusion itself consists of data preparation and data registration. A challenge in data fusion is the different resolutions of sensors. Therefore, up- and downsampling algorithms are needed. This paper compares multiple up- and downsampling methods, such as different direct interpolation methods, joint bilateral upsampling (JBU), and Markov random fields (MRFs), in terms of their potential to create RGB-D images and improve the quality of depth information. In contrast to the literature in which imaging systems are adjusted to acquire the data of the same section simultaneously, the laboratory setup in this study was based on conveyor-based optical sorting processes, and therefore, the data were acquired at different time periods and different spatial locations. Data assignment and data cropping were necessary. In order to evaluate the results, root mean square error (RMSE), signal-to-noise ratio (SNR), correlation (CORR), universal quality index (UQI), and the contour offset are monitored. With JBU outperforming the other upsampling methods, achieving a meanRMSE = 25.22, mean SNR = 32.80, mean CORR = 0.99, and mean UQI = 0.97.
自3D传感器普及以来，在消费领域获取成像深度数据变得更加容易。在诸如工业物体缺陷定位或质量/体积估计等应用中，精确的深度数据至关重要，因此受益于多种信息源的使用。然而，RGB图像和深度图像的结合不仅可以增进我们对物体的理解，使人们能够获取更多关于物体的信息，还能提高数据质量。使用数据融合组合不同的相机系统可以产生更高质量的数据，因为缺点可以得到补偿。数据融合本身包括数据准备和数据配准。数据融合中的一个挑战是传感器的不同分辨率。因此，需要上采样和下采样算法。本文比较了多种上采样和下采样方法，如不同的直接插值方法、联合双边向上采样（JBU）和马尔可夫随机场（MRF），就它们创建RGB-D图像和提高深度信息质量的潜力而言。与文献中调整成像系统以同时获取同一截面的数据不同，本研究中的实验室设置基于基于传送带的光学分拣过程，因此，数据是在不同的时间段和不同的空间位置获取的。数据分配和数据裁剪是必要的。为了评估结果，监测了均方根误差（RMSE）、信噪比（SNR）、相关性（CORR）、通用质量指数（UQI）和轮廓偏移。JBU的表现优于其他上采样方法，平均RMSE = 25.22，平均SNR = 32.80，平均CORR = 0.99，平均UQI = 0.97。
