Jun Woomin, Yoo Jisang, Lee Sungjin
Electronic Engineering, Dong Seoul University, Seongnam 13117, Republic of Korea.
Autonomous Driving Lab, Modulabs, Seoul 06252, Republic of Korea.
Sensors (Basel). 2024 Jun 28;24(13):4205. doi: 10.3390/s24134205.
Accurate 3D image recognition, critical for autonomous driving safety, is shifting from the LIDAR-based point cloud to camera-based depth estimation technologies driven by cost considerations and the point cloud's limitations in detecting distant small objects. This research aims to enhance MDE (Monocular Depth Estimation) using a single camera, offering extreme cost-effectiveness in acquiring 3D environmental data. In particular, this paper focuses on novel data augmentation methods designed to enhance the accuracy of MDE. Our research addresses the challenge of limited MDE data quantities by proposing the use of synthetic-based augmentation techniques: Mask, Mask-Scale, and CutFlip. The implementation of these synthetic-based data augmentation strategies has demonstrably enhanced the accuracy of MDE models by 4.0% compared to the original dataset. Furthermore, this study introduces the RMS (Real-time Monocular Depth Estimation configuration considering Resolution, Efficiency, and Latency) algorithm, designed for the optimization of neural networks to augment the performance of contemporary monocular depth estimation technologies through a three-step process. Initially, it selects a model based on minimum latency and REL criteria, followed by refining the model's accuracy using various data augmentation techniques and loss functions. Finally, the refined model is compressed using quantization and pruning techniques to minimize its size for efficient on-device real-time applications. Experimental results from implementing the RMS algorithm indicated that, within the required latency and size constraints, the IEBins model exhibited the most accurate REL (absolute RELative error) performance, achieving a 0.0480 REL. Furthermore, the data augmentation combination of the original dataset with Flip, Mask, and CutFlip, alongside the loss function, displayed the best REL performance, with a score of 0.0461. The network compression technique using FP16 was analyzed as the most effective, reducing the model size by 83.4% compared to the original while maintaining the least impact on REL performance and latency. Finally, the performance of the RMS algorithm was validated on the on-device autonomous driving platform, NVIDIA Jetson AGX Orin, through which optimal deployment strategies were derived for various applications and scenarios requiring autonomous driving technologies.
精确的3D图像识别对自动驾驶安全至关重要，由于成本因素以及点云在检测远处小物体方面的局限性，它正从基于激光雷达的点云转向基于摄像头的深度估计技术。本研究旨在使用单摄像头增强单目深度估计（MDE），在获取3D环境数据方面提供极高的成本效益。特别是，本文重点关注旨在提高MDE准确性的新型数据增强方法。我们的研究通过提出使用基于合成的增强技术（掩码、掩码比例和裁剪翻转）来应对MDE数据量有限的挑战。与原始数据集相比，这些基于合成的数据增强策略的实施已将MDE模型的准确性显著提高了4.0%。此外，本研究引入了RMS（考虑分辨率、效率和延迟的实时单目深度估计配置）算法，该算法旨在通过三个步骤优化神经网络，以增强当代单目深度估计技术的性能。首先，它根据最小延迟和REL标准选择模型，然后使用各种数据增强技术和损失函数提高模型的准确性。最后，使用量化和剪枝技术对优化后的模型进行压缩，以最小化其大小，实现高效的设备上实时应用。实施RMS算法的实验结果表明，在所需的延迟和大小限制内，IEBins模型表现出最准确的REL（绝对相对误差）性能，达到0.048左右。此外，原始数据集与翻转、掩码和裁剪翻转的数据增强组合以及损失函数显示出最佳的REL性能，得分为0.0461。分析得出使用FP16的网络压缩技术最为有效，与原始模型相比，模型大小减少了83.4%，同时对REL性能和延迟的影响最小。最后，通过在设备上的自动驾驶平台NVIDIA Jetson AGX Orin上验证了RMS算法的性能，从中得出了针对各种需要自动驾驶技术的应用和场景的最佳部署策略。
