Wang Yuhao, Zuo Yong, Du Zhihua, Song Xiaohan, Luo Tian, Hong Xiaobin, Wu Jian
School of Electronic Engineering, Beijing University of Post and Telecommunications, Beijing 100876, China.
Sensors (Basel). 2023 Jul 12;23(14):6327. doi: 10.3390/s23146327.
Three-dimensional LiDAR systems that capture point cloud data enable the simultaneous acquisition of spatial geometry and multi-wavelength intensity information, thereby paving the way for three-dimensional point cloud recognition and processing. However, due to the irregular distribution, low resolution of point clouds, and limited spatial recognition accuracy in complex environments, inherent errors occur in classifying and segmenting the acquired target information. Conversely, two-dimensional visible light images provide real-color information, enabling the distinction of object contours and fine details, thus yielding clear, high-resolution images when desired. The integration of two-dimensional information with point clouds offers complementary advantages. In this paper, we present the incorporation of two-dimensional information to form a multi-modal representation. From this, we extract local features to establish three-dimensional geometric relationships and two-dimensional color relationships. We introduce a novel network model, termed MInet (Multi-Information net), which effectively captures features relating to both two-dimensional color and three-dimensional pose information. This enhanced network model improves feature saliency, thereby facilitating superior segmentation and recognition tasks. We evaluate our MInet architecture using the ShapeNet and ThreeDMatch datasets for point cloud segmentation, and the Stanford dataset for object recognition. The robust results, coupled with quantitative and qualitative experiments, demonstrate the superior performance of our proposed method in point cloud segmentation and object recognition tasks.
能够捕获点云数据的三维激光雷达系统可同时获取空间几何信息和多波长强度信息，从而为三维点云识别与处理铺平道路。然而，由于点云分布不规则、分辨率低以及在复杂环境中空间识别精度有限，在对获取的目标信息进行分类和分割时会出现固有误差。相反，二维可见光图像提供真实颜色信息，能够区分物体轮廓和精细细节，从而在需要时生成清晰、高分辨率的图像。将二维信息与点云相结合具有互补优势。在本文中，我们展示了将二维信息合并以形成多模态表示。由此，我们提取局部特征以建立三维几何关系和二维颜色关系。我们引入了一种新颖的网络模型，称为MInet（多信息网络），它能有效捕获与二维颜色和三维姿态信息相关的特征。这种增强的网络模型提高了特征显著性，从而有助于完成更出色的分割和识别任务。我们使用ShapeNet和ThreeDMatch数据集进行点云分割，并使用斯坦福数据集进行目标识别来评估我们的MInet架构。稳健的结果以及定量和定性实验表明，我们提出的方法在点云分割和目标识别任务中具有卓越性能。
