Kim Sehyeon, Jeong Tae-In, Kim San, Choi Eunji, Yang Eunju, Song Munki, Eom Tae Joong, Kim Chang-Seok, Gliserin Alexander, Kim Seungchul
Department of Cogno-Mechatronics Engineering, College of Nanoscience and Nanotechnology, Pusan National University, Busan, 46241, Republic of Korea.
Department of Optics and Mechatronics Engineering, College of Nanoscience and Nanotechnology, Pusan National University, Busan, 46241, Republic of Korea.
Sci Rep. 2024 May 20;14(1):11445. doi: 10.1038/s41598-024-62342-2.
The recent progress in the development of measurement systems for autonomous recognition had a substantial impact on emerging technology in numerous fields, especially robotics and automotive applications. In particular, time-of-flight (TOF) based light detection and ranging (LiDAR) systems enable to map the surrounding environmental information over long distances and with high accuracy. The combination of advanced LiDAR with an artificial intelligence platform allows enhanced object recognition and classification, which however still suffers from limitations of inaccuracy and misidentification. Recently, multi-spectral LiDAR systems have been employed to increase the object recognition performance by additionally providing material information in the short-wave infrared (SWIR) range where the reflection spectrum characteristics are typically very sensitive to material properties. However, previous multi-spectral LiDAR systems utilized band-pass filters or complex dispersive optical systems and even required multiple photodetectors, adding complexity and cost. In this work, we propose a time-division-multiplexing (TDM) based multi-spectral LiDAR system for semantic object inference by the simultaneous acquisition of spatial and spectral information. By utilizing the TDM method, we enable the simultaneous acquisition of spatial and spectral information as well as a TOF based distance map with minimized optical loss using only a single photodetector. Our LiDAR system utilizes nanosecond pulses of five different wavelengths in the SWIR range to acquire sufficient material information in addition to 3D spatial information. To demonstrate the recognition performance, we map the multi-spectral image from a human hand, a mannequin hand, a fabric gloved hand, a nitrile gloved hand, and a printed human hand onto an RGB-color encoded image, which clearly visualizes spectral differences as RGB color depending on the material while having a similar shape. Additionally, the classification performance of the multi-spectral image is demonstrated with a convolution neural network (CNN) model using the full multi-spectral data set. Our work presents a compact novel spectroscopic LiDAR system, which provides increased recognition performance and thus a great potential to improve safety and reliability in autonomous driving.
自主识别测量系统开发方面的最新进展对众多领域的新兴技术产生了重大影响，尤其是机器人技术和汽车应用领域。特别是，基于飞行时间（TOF）的光探测与测距（LiDAR）系统能够高精度地远距离绘制周围环境信息。先进的LiDAR与人工智能平台相结合，可以增强目标识别和分类能力，然而，这仍然存在不准确和误识别的局限性。最近，多光谱LiDAR系统已被用于通过在短波红外（SWIR）范围内额外提供材料信息来提高目标识别性能，在该范围内反射光谱特性通常对材料属性非常敏感。然而，以前的多光谱LiDAR系统使用带通滤波器或复杂的色散光学系统，甚至需要多个光电探测器，这增加了复杂性和成本。在这项工作中，我们提出了一种基于时分复用（TDM）的多光谱LiDAR系统，用于通过同时获取空间和光谱信息进行语义目标推理。通过利用TDM方法，我们仅使用单个光电探测器就能够同时获取空间和光谱信息以及基于TOF的距离图，同时将光损耗降至最低。我们的LiDAR系统利用SWIR范围内五个不同波长的纳秒脉冲，除了获取3D空间信息外，还能获取足够的材料信息。为了展示识别性能，我们将来自人手模型、人体模型手、戴织物手套的手、戴丁腈手套的手和打印人手的多光谱图像映射到RGB颜色编码图像上，该图像根据材料将光谱差异清晰地可视化为RGB颜色，同时具有相似的形状。此外，使用卷积神经网络（CNN）模型对完整的多光谱数据集展示了多光谱图像的分类性能。我们的工作展示了一种紧凑新颖的光谱LiDAR系统，它提供了更高的识别性能，因此在提高自动驾驶的安全性和可靠性方面具有巨大潜力。
