Geng Ruixu, Li Yadong, Zhang Dongheng, Wu Jincheng, Gao Yating, Hu Yang, Chen Yan
IEEE Trans Image Process. 2024 Dec 11;PP. doi: 10.1109/TIP.2024.3512356.
Millimeter-wave (mmWave) radar pointcloud offers attractive potential for 3D sensing, thanks to its robustness in challenging conditions such as smoke and low illumination. However, existing methods failed to simultaneously address the three main challenges in mmWave radar pointcloud reconstruction: specular information lost, low angular resolution, and severe interference. In this paper, we propose DREAM-PCD, a novel framework specifically designed for real-time 3D environment sensing that combines signal processing and deep learning methods into three well-designed components to tackle all three challenges: Non-Coherent Accumulation for dense points, Synthetic Aperture Accumulation for improved angular resolution, and Real-Denoise Multiframe network for interference removal. By leveraging causal multiple viewpoints accumulation and the "real-denoise" mechanism, DREAM-PCD significantly enhances the generalization performance and real-time capability. We also introduce RadarEyes, the largest mmWave indoor dataset with over 1,000,000 frames, featuring a unique design incorporating two orthogonal single-chip radars, Lidar, and camera, enriching dataset diversity and applications. Experimental results demonstrate that DREAM-PCD surpasses existing methods in reconstruction quality, and exhibits superior generalization and real-time capabilities, enabling high-quality real-time reconstruction of radar pointcloud under various parameters and scenarios. We believe that DREAM-PCD, along with the RadarEyes dataset, will significantly advance mmWave radar perception in future real-world applications.
毫米波（mmWave）雷达点云在3D传感方面具有诱人的潜力，这得益于其在烟雾和低光照等具有挑战性的条件下的稳健性。然而，现有方法未能同时解决毫米波雷达点云重建中的三个主要挑战：镜面信息丢失、低角分辨率和严重干扰。在本文中，我们提出了DREAM-PCD，这是一个专门为实时3D环境传感设计的新颖框架，它将信号处理和深度学习方法整合到三个精心设计的组件中，以应对所有这三个挑战：用于密集点的非相干积累、用于提高角分辨率的合成孔径积累以及用于去除干扰的实域去噪多帧网络。通过利用因果多视角积累和“实域去噪”机制，DREAM-PCD显著提高了泛化性能和实时能力。我们还引入了RadarEyes，这是最大的毫米波室内数据集，拥有超过100万帧，其独特设计结合了两个正交单芯片雷达、激光雷达和相机，丰富了数据集的多样性和应用。实验结果表明，DREAM-PCD在重建质量上超越了现有方法，并展现出卓越的泛化和实时能力，能够在各种参数和场景下对雷达点云进行高质量的实时重建。我们相信，DREAM-PCD连同RadarEyes数据集，将在未来的实际应用中显著推动毫米波雷达感知技术的发展。
