School of Optical-Electrical and Computer Engineering, University of Shanghai for Science & Technology, Shanghai, 200093, China.
Sci Rep. 2022 Jul 5;12(1):11320. doi: 10.1038/s41598-022-14438-w.
Reliable road detection is an essential task in autonomous driving systems. Two categories of sensors are commonly used, cameras and light detection and ranging (LiDAR), each of which can provide corresponding supplements. Nevertheless, existing sensor fusion methods do not fully utilize multimodal data. Most of them are dominated by images and take point clouds as a supplement rather than making the best of them, and the correlation between modalities is ignored. This paper proposes a recurrent conditional random field (R-CRF) model to fuse images and point clouds for road detection. The R-CRF model integrates results (information) from modalities in a probabilistic way. Each modality is independently processed with its semantic segmentation network. The probability scores obtained are considered a unary term for individual pixel nodes in a random field, while RGB images and the densified LiDAR images are used as pairwise terms. The energy function is then iteratively optimized by mean-field variational inference, and the labelling results are refined by exploiting fully connected graphs of the RGB image and LiDAR images. Extensive experiments are conducted on the public KITTI-Road dataset, and the proposed method achieves competitive performance.
可靠的道路检测是自动驾驶系统中的一项基本任务。通常使用两类传感器，摄像机和光探测与测距（LiDAR），它们各自可以提供相应的补充。然而，现有的传感器融合方法并没有充分利用多模态数据。大多数方法以图像为主，将点云作为补充，而不是充分利用它们，并且忽略了模态之间的相关性。本文提出了一种用于道路检测的融合图像和点云的递归条件随机场（R-CRF）模型。R-CRF 模型以概率方式集成来自各模态的结果（信息）。每个模态都使用其语义分割网络进行独立处理。所获得的概率分数被视为随机场中各个像素节点的一元项，而 RGB 图像和密集化的 LiDAR 图像则被用作对项。然后通过平均场变分推断迭代优化能量函数，并通过利用 RGB 图像和 LiDAR 图像的全连接图来细化标记结果。在公共 KITTI-Road 数据集上进行了广泛的实验，所提出的方法具有竞争性能。
