Opt Express. 2023 Feb 27;31(5):7060-7072. doi: 10.1364/OE.478308.
3D time-of-flight (ToF) image sensors are used widely in applications such as self-driving cars, augmented reality (AR), and robotics. When implemented with single-photon avalanche diodes (SPADs), compact, array format sensors can be made that offer accurate depth maps over long distances, without the need for mechanical scanning. However, array sizes tend to be small, leading to low lateral resolution, which combined with low signal-to-background ratio (SBR) levels under high ambient illumination, may lead to difficulties in scene interpretation. In this paper, we use synthetic depth sequences to train a 3D convolutional neural network (CNN) for denoising and upscaling (×4) depth data. Experimental results, based on synthetic as well as real ToF data, are used to demonstrate the effectiveness of the scheme. With GPU acceleration, frames are processed at >30 frames per second, making the approach suitable for low-latency imaging, as required for obstacle avoidance.
3D 飞行时间 (ToF) 图像传感器广泛应用于自动驾驶汽车、增强现实 (AR) 和机器人等领域。当与单光子雪崩二极管 (SPAD) 结合使用时，可以制造出紧凑的阵列格式传感器，这些传感器可以在长距离内提供准确的深度图，而无需机械扫描。然而，阵列尺寸往往较小，导致横向分辨率较低，再加上在高环境光照下的低信号与背景比 (SBR) 水平，可能会导致场景解释困难。在本文中，我们使用合成深度序列来训练 3D 卷积神经网络 (CNN) 进行去噪和上采样 (×4) 深度数据。基于合成和真实 ToF 数据的实验结果证明了该方案的有效性。通过 GPU 加速，每秒处理帧数超过 30 帧，因此该方法适用于障碍物避免等低延迟成像。
