Winter Jakub, Nowak Robert
Institute of Computer Science, Warsaw University of Technology, Nowowiejska 15/19, 00-665 Warsaw, Poland.
Sensors (Basel). 2024 Sep 5;24(17):5786. doi: 10.3390/s24175786.
Fusing data from many sources helps to achieve improved analysis and results. In this work, we present a new algorithm to fuse data from multiple cameras with data from multiple lidars. This algorithm was developed to increase the sensitivity and specificity of autonomous vehicle perception systems, where the most accurate sensors measuring the vehicle's surroundings are cameras and lidar devices. Perception systems based on data from one type of sensor do not use complete information and have lower quality. The camera provides two-dimensional images; lidar produces three-dimensional point clouds. We developed a method for matching pixels on a pair of stereoscopic images using dynamic programming inspired by an algorithm to match sequences of amino acids used in bioinformatics. We improve the quality of the basic algorithm using additional data from edge detectors. Furthermore, we also improve the algorithm performance by reducing the size of matched pixels determined by available car speeds. We perform point cloud densification in the final step of our method, fusing lidar output data with stereo vision output. We implemented our algorithm in C++ with Python API, and we provided the open-source library named Stereo PCD. This library very efficiently fuses data from multiple cameras and multiple lidars. In the article, we present the results of our approach to benchmark databases in terms of quality and performance. We compare our algorithm with other popular methods.
融合来自多个源的数据有助于实现更好的分析和结果。在这项工作中，我们提出了一种新算法，用于将来自多个摄像头的数据与来自多个激光雷达的数据进行融合。开发该算法是为了提高自动驾驶车辆感知系统的灵敏度和特异性，其中测量车辆周围环境最准确的传感器是摄像头和激光雷达设备。基于一种类型传感器数据的感知系统没有使用完整信息，质量较低。摄像头提供二维图像；激光雷达生成三维点云。我们受生物信息学中用于匹配氨基酸序列的算法启发，利用动态规划开发了一种在一对立体图像上匹配像素的方法。我们使用来自边缘检测器的额外数据提高了基本算法的质量。此外，我们还通过根据可用车速减小匹配像素的大小来提高算法性能。在我们方法的最后一步，我们将激光雷达输出数据与立体视觉输出进行融合，实现点云致密化。我们用C++和Python API实现了我们的算法，并提供了名为Stereo PCD的开源库。该库非常高效地融合了来自多个摄像头和多个激光雷达的数据。在本文中，我们从质量和性能方面展示了我们对基准数据库的方法的结果。我们将我们的算法与其他流行方法进行了比较。
