Chen Duo, Sang Xinzhu, Peng Wang, Yu Xunbo, Wang Hua Chun
Opt Express. 2018 Oct 15;26(21):27585-27598. doi: 10.1364/OE.26.027585.
Multi-view applications have been used in a wide range, especially Three-Dimensional (3D) display. Since capturing dense multiple views for 3D light-field display is still a difficult work, view synthesis becomes an accessible way. Convolutional neural networks (CNN) has been used to synthesize new views of the scene. However, training targets are sometimes difficult to obtain, and it views are very difficult to synthesize at arbitrary positions. Here, an unsupervised network of Multi-Parallax View Net (MPVN) is proposed, which can synthesize multi-parallax views for 3D light-field display. Existing parallax views are re-projected to the target position to build input towers. The network is operated on these towers, and outputs a color tower and a selection tower. These two towers yield the final output image by per-pixel weight summing. MPVN adopts end-to-end unsupervised training to minimize prediction errors at existing positions. It can predict virtual views at any parallax position between existing views in a high quality. Experimental results demonstrate the validation of our proposed network, and SSIM of synthetic views are mostly over 0.95. We believe that this method can effectively provide enough views for 3D light-field display in the future work.
多视图应用已经在广泛的领域中得到应用，尤其是在三维（3D）显示方面。由于为3D光场显示捕获密集的多视图仍然是一项艰巨的工作，视图合成成为一种可行的方法。卷积神经网络（CNN）已被用于合成场景的新视图。然而，训练目标有时难以获得，并且在任意位置合成视图非常困难。在此，提出了一种无监督的多视差视图网络（MPVN），它可以为3D光场显示合成多视差视图。将现有的视差视图重新投影到目标位置以构建输入塔。网络在这些塔上运行，并输出一个颜色塔和一个选择塔。这两个塔通过逐像素加权求和产生最终输出图像。MPVN采用端到端无监督训练以最小化现有位置的预测误差。它可以高质量地预测现有视图之间任意视差位置的虚拟视图。实验结果证明了我们提出的网络的有效性，合成视图的结构相似性指数（SSIM）大多超过0.95。我们相信这种方法可以在未来的工作中有效地为3D光场显示提供足够的视图。
