Wen Yang, Sheng Bin, Li Ping, Lin Weiyao, Feng David Dagan
IEEE Trans Image Process. 2018 Oct 8. doi: 10.1109/TIP.2018.2874285.
Depth image super-resolution is a significant yet challenging task. In this paper, we introduce a novel deep color guided coarse-to-fine convolutional neural network (CNN) framework to address this problem. First, we present a datadriven filter method to approximate the ideal filter for depth image super-resolution instead of hand-designed filters. Based on large data samples, the filter learned is more accurate and stable for upsampling depth image. Second, we introduce a coarse-to-fine CNN to learn different sizes of filter kernels. In coarse stage, larger filter kernels are learned by CNN to achieve crude high-resolution depth image. As to fine stage, the crude high-resolution depth image is used as the input so that smaller filter kernels are learned to gain more accurate results. Benefit from this network, we can progressively recover the high frequency details. Third, we construct a color guidance strategy that fuses color difference and spatial distance for depth image upsampling. We revise the interpolated high-resolution depth image according to the corresponding pixels in highresolution color maps. Guided by color information, the depth of high-resolution image obtained can alleviate texture copying artifacts and preserve edge details effectively. Quantitative and qualitative experimental results demonstrate our state-of-the-art performance for depth map super-resolution.
深度图像超分辨率是一项重要但具有挑战性的任务。在本文中，我们引入了一种新颖的深度颜色引导的粗到细卷积神经网络（CNN）框架来解决这个问题。首先，我们提出一种数据驱动的滤波方法，以近似用于深度图像超分辨率的理想滤波器，而不是手工设计的滤波器。基于大量数据样本，所学习的滤波器对于深度图像的上采样更加准确和稳定。其次，我们引入一个粗到细的CNN来学习不同大小的滤波器内核。在粗阶段，CNN学习较大的滤波器内核以获得粗糙的高分辨率深度图像。至于细阶段，将粗糙的高分辨率深度图像用作输入，以便学习较小的滤波器内核以获得更准确的结果。受益于这个网络，我们可以逐步恢复高频细节。第三，我们构建一种颜色引导策略，该策略融合颜色差异和空间距离用于深度图像上采样。我们根据高分辨率颜色图中的相应像素来修正插值后的高分辨率深度图像。在颜色信息的引导下，所获得的高分辨率图像的深度可以减轻纹理复制伪影并有效地保留边缘细节。定量和定性实验结果证明了我们在深度图超分辨率方面的领先性能。
