Oh Sang-Il, Kang Hang-Bong
Department of Media Engineering, Catholic University of Korea, 43-1, Yeoggok 2-dong, Wonmmi-gu, Bucheon-si, Gyeonggi-do 14662, Korea.
Sensors (Basel). 2017 Jan 22;17(1):207. doi: 10.3390/s17010207.
To understand driving environments effectively, it is important to achieve accurate detection and classification of objects detected by sensor-based intelligent vehicle systems, which are significantly important tasks. Object detection is performed for the localization of objects, whereas object classification recognizes object classes from detected object regions. For accurate object detection and classification, fusing multiple sensor information into a key component of the representation and perception processes is necessary. In this paper, we propose a new object-detection and classification method using decision-level fusion. We fuse the classification outputs from independent unary classifiers, such as 3D point clouds and image data using a convolutional neural network (CNN). The unary classifiers for the two sensors are the CNN with five layers, which use more than two pre-trained convolutional layers to consider local to global features as data representation. To represent data using convolutional layers, we apply region of interest (ROI) pooling to the outputs of each layer on the object candidate regions generated using object proposal generation to realize color flattening and semantic grouping for charge-coupled device and Light Detection And Ranging (LiDAR) sensors. We evaluate our proposed method on a KITTI benchmark dataset to detect and classify three object classes: cars, pedestrians and cyclists. The evaluation results show that the proposed method achieves better performance than the previous methods. Our proposed method extracted approximately 500 proposals on a 1226 × 370 image, whereas the original selective search method extracted approximately 10 6 × n proposals. We obtained classification performance with 77.72% mean average precision over the entirety of the classes in the moderate detection level of the KITTI benchmark dataset.
为了有效理解驾驶环境，实现基于传感器的智能车辆系统所检测到物体的准确检测和分类至关重要，这是非常重要的任务。目标检测用于物体的定位，而目标分类则从检测到的物体区域识别物体类别。为了进行准确的目标检测和分类，将多个传感器信息融合到表示和感知过程的关键组件中是必要的。在本文中，我们提出了一种使用决策级融合的新目标检测和分类方法。我们使用卷积神经网络（CNN）融合来自独立一元分类器的分类输出，例如三维点云和图像数据。这两个传感器的一元分类器是具有五层的CNN，其使用超过两个预训练的卷积层来将局部到全局特征作为数据表示。为了使用卷积层表示数据，我们将感兴趣区域（ROI）池化应用于使用目标提议生成生成的目标候选区域上各层的输出，以实现电荷耦合器件和激光雷达（LiDAR）传感器的颜色扁平化和语义分组。我们在KITTI基准数据集上评估我们提出的方法，以检测和分类三类物体：汽车、行人及骑自行车的人。评估结果表明，所提出的方法比以前的方法具有更好的性能。我们提出的方法在1226×370的图像上提取了大约500个提议，而原始的选择性搜索方法提取了大约106×n个提议。在KITTI基准数据集的中等检测水平下，我们在所有类别的整体上获得了平均精度为77.72%的分类性能。
