IEEE Trans Image Process. 2017 Jan;26(1):315-327. doi: 10.1109/TIP.2016.2612826.
One of the most challenging issues in color guided depth map restoration is the inconsistency between color edges in guidance color images and depth discontinuities on depth maps. This makes the restored depth map suffer from texture copy artifacts and blurring depth discontinuities. To handle this problem, most state-of-the-art methods design complex guidance weight based on guidance color images and heuristically make use of the bicubic interpolation of the input depth map. In this paper, we show that using bicubic interpolated depth map can blur depth discontinuities when the upsampling factor is large and the input depth map contains large holes and heavy noise. In contrast, we propose a robust optimization framework for color guided depth map restoration. By adopting a robust penalty function to model the smoothness term of our model, we show that the proposed method is robust against the inconsistency between color edges and depth discontinuities even when we use simple guidance weight. To the best of our knowledge, we are the first to solve this problem with a principled mathematical formulation rather than previous heuristic weighting schemes. The proposed robust method performs well in suppressing texture copy artifacts. Moreover, it can better preserve sharp depth discontinuities than previous heuristic weighting schemes. Through comprehensive experiments on both simulated data and real data, we show promising performance of the proposed method.
在基于颜色的深度图恢复中，最具挑战性的问题之一是引导颜色图像中的颜色边缘和深度图上的深度不连续性之间的不一致性。这使得恢复的深度图受到纹理复制伪影和深度不连续性模糊的影响。为了解决这个问题，大多数最先进的方法基于引导颜色图像设计复杂的引导权重，并启发式地利用输入深度图的双三次插值。在本文中，我们表明，当上采样因子较大且输入深度图包含大孔和重噪声时，使用双三次插值的深度图会模糊深度不连续性。相比之下，我们提出了一种用于基于颜色的深度图恢复的稳健优化框架。通过采用鲁棒惩罚函数来对我们模型的平滑项进行建模，我们表明，即使使用简单的引导权重，所提出的方法也能对颜色边缘和深度不连续性之间的不一致性具有鲁棒性。据我们所知，我们是第一个用有原则的数学公式而不是以前的启发式加权方案来解决这个问题的。所提出的稳健方法在抑制纹理复制伪影方面表现良好。此外，与以前的启发式加权方案相比，它可以更好地保留锐利的深度不连续性。通过对模拟数据和真实数据的综合实验，我们展示了所提出的方法具有有前景的性能。
