Bertel Tobias, Campbell Neill D F, Richardt Christian
IEEE Trans Vis Comput Graph. 2019 May;25(5):1828-1835. doi: 10.1109/TVCG.2019.2898799. Epub 2019 Feb 25.
The ubiquity of smart mobile devices, such as phones and tablets, enables users to casually capture 360° panoramas with a single camera sweep to share and relive experiences. However, panoramas lack motion parallax as they do not provide different views for different viewpoints. The motion parallax induced by translational head motion is a crucial depth cue in daily life. Alternatives, such as omnidirectional stereo panoramas, provide different views for each eye (binocular disparity), but they also lack motion parallax as the left and right eye panoramas are stitched statically. Methods based on explicit scene geometry reconstruct textured 3D geometry, which provides motion parallax, but suffers from visible reconstruction artefacts. The core of our method is a novel multi-perspective panorama representation, which can be casually captured and rendered with motion parallax for each eye on the fly. This provides a more realistic perception of panoramic environments which is particularly useful for virtual reality applications. Our approach uses a single consumer video camera to acquire 200-400 views of a real 360° environment with a single sweep. By using novel-view synthesis with flow-based blending, we show how to turn these input views into an enriched 360° panoramic experience that can be explored in real time, without relying on potentially unreliable reconstruction of scene geometry. We compare our results with existing omnidirectional stereo and image-based rendering methods to demonstrate the benefit of our approach, which is the first to enable casual consumers to capture and view high-quality 360° panoramas with motion parallax.
诸如手机和平板电脑之类的智能移动设备的普及，使得用户能够通过单次相机扫描轻松捕捉360°全景，以分享和重温体验。然而，全景图缺乏运动视差，因为它们不会为不同的视点提供不同的视图。平移头部运动引起的运动视差是日常生活中至关重要的深度线索。诸如全向立体全景图之类的替代方案为每只眼睛提供不同的视图（双目视差），但它们也缺乏运动视差，因为左右眼全景图是静态拼接的。基于显式场景几何的方法重建有纹理的3D几何，它提供运动视差，但会出现可见的重建伪像。我们方法的核心是一种新颖的多视角全景表示，它可以轻松捕捉并实时为每只眼睛渲染运动视差。这提供了对全景环境更逼真的感知，这对于虚拟现实应用特别有用。我们的方法使用单个消费级摄像机通过单次扫描获取真实360°环境的200 - 400个视图。通过使用基于流混合的新视图合成，我们展示了如何将这些输入视图转化为丰富的360°全景体验，可实时探索，而无需依赖可能不可靠的场景几何重建。我们将我们的结果与现有的全向立体和基于图像的渲染方法进行比较，以证明我们方法的优势，该方法是第一个使普通消费者能够捕捉和观看具有运动视差的高质量360°全景图的方法。
