Faculty of Materials and Manufacturing, Beijing University of Technology, 100 Pingleyuan, Chaoyang District, Beijing, 100124, China.
Sci Rep. 2023 May 17;13(1):8056. doi: 10.1038/s41598-023-35170-z.
Autonomous driving has been widely applied in commercial and industrial applications, along with the upgrade of environmental awareness systems. Tasks such as path planning, trajectory tracking, and obstacle avoidance are strongly dependent on the ability to perform real-time object detection and position regression. Among the most commonly used sensors, camera provides dense semantic information but lacks accurate distance information to the target, while LiDAR provides accurate depth information but with sparse resolution. In this paper, a LiDAR-camera-based fusion algorithm is proposed to improve the above-mentioned trade-off problems by constructing a Siamese network for object detection. Raw point clouds are converted to camera planes to obtain a 2D depth image. By designing a cross feature fusion block to connect the depth and RGB processing branches, the feature-layer fusion strategy is applied to integrate multi-modality data. The proposed fusion algorithm is evaluated on the KITTI dataset. Experimental results demonstrate that our algorithm has superior performance and real-time efficiency. Remarkably, it outperforms other state-of-the-art algorithms at the most important moderate level and achieves excellent performance at the easy and hard levels.
自动驾驶已经广泛应用于商业和工业领域，同时环境意识系统也在不断升级。路径规划、轨迹跟踪和障碍物回避等任务强烈依赖于实时目标检测和位置回归的能力。在最常用的传感器中，相机提供了密集的语义信息，但缺乏对目标的精确距离信息，而激光雷达则提供了精确的深度信息，但分辨率稀疏。本文提出了一种基于激光雷达-相机融合的算法，通过构建一个用于目标检测的孪生网络，来改善上述权衡问题。原始点云被转换到相机平面，以获得二维深度图像。通过设计一个交叉特征融合块来连接深度和 RGB 处理分支，应用特征层融合策略来整合多模态数据。该融合算法在 KITTI 数据集上进行了评估。实验结果表明，我们的算法具有优越的性能和实时效率。值得注意的是，它在最重要的中等水平上优于其他最先进的算法，并在简单和困难水平上取得了优异的性能。
