IEEE Trans Vis Comput Graph. 2017 Nov;23(11):2455-2462. doi: 10.1109/TVCG.2017.2734578. Epub 2017 Aug 10.
We present a real-time method for rendering novel virtual camera views from given RGB-D (color and depth) data of a different viewpoint. Missing color and depth information due to incomplete input or disocclusions is efficiently inpainted in a temporally consistent way. The inpainting takes the location of strong image gradients into account as likely depth discontinuities. We present our method in the context of a view correction system for mobile devices, and discuss how to obtain a screen-camera calibration and options for acquiring depth input. Our method has use cases in both augmented and virtual reality applications. We demonstrate the speed of our system and the visual quality of its results in multiple experiments in the paper as well as in the supplementary video.
我们提出了一种从给定的不同视点的 RGB-D（颜色和深度）数据实时渲染新的虚拟摄像机视图的方法。由于输入不完整或遮挡而缺失的颜色和深度信息以时间一致的方式进行有效修复。修复考虑了图像梯度的位置，因为这些位置可能存在深度不连续。我们在移动设备的视图校正系统的上下文中介绍了我们的方法，并讨论了如何获取屏幕-摄像机校准以及获取深度输入的选项。我们的方法在增强现实和虚拟现实应用中都有应用场景。我们在论文中的多个实验以及补充视频中展示了我们系统的速度及其结果的视觉质量。
