Im Sunghoon, Ha Hyowon, Choe Gyeongmin, Jeon Hae-Gon, Joo Kyungdon, Kweon In So
IEEE Trans Pattern Anal Mach Intell. 2019 Apr;41(4):775-787. doi: 10.1109/TPAMI.2018.2819679. Epub 2018 Mar 26.
Structure from small motion has become an important topic in 3D computer vision as a method for estimating depth, since capturing the input is so user-friendly. However, major limitations exist with respect to the form of depth uncertainty, due to the narrow baseline and the rolling shutter effect. In this paper, we present a dense 3D reconstruction method from small motion clips using commercial hand-held cameras, which typically cause the undesired rolling shutter artifact. To address these problems, we introduce a novel small motion bundle adjustment that effectively compensates for the rolling shutter effect. Moreover, we propose a pipeline for a fine-scale dense 3D reconstruction that models the rolling shutter effect by utilizing both sparse 3D points and the camera trajectory from narrow-baseline images. In this reconstruction, the sparse 3D points are propagated to obtain an initial depth hypothesis using a geometry guidance term. Then, the depth information on each pixel is obtained by sweeping the plane around each depth search space near the hypothesis. The proposed framework shows accurate dense reconstruction results suitable for various sought-after applications. Both qualitative and quantitative evaluations show that our method consistently generates better depth maps compared to state-of-the-art methods.
基于小运动的结构恢复作为一种估计深度的方法，在三维计算机视觉中已成为一个重要课题，因为其输入的获取非常方便用户使用。然而，由于基线狭窄和卷帘快门效应，在深度不确定性形式方面存在重大限制。在本文中，我们提出了一种使用商用手持相机从小运动片段进行密集三维重建的方法，这种相机通常会产生不期望的卷帘快门伪影。为了解决这些问题，我们引入了一种新颖的小运动光束法平差，它能有效补偿卷帘快门效应。此外，我们提出了一种用于精细尺度密集三维重建的流程，该流程通过利用稀疏三维点和来自窄基线图像的相机轨迹对卷帘快门效应进行建模。在这种重建中，利用几何引导项传播稀疏三维点以获得初始深度假设。然后，通过在假设附近的每个深度搜索空间周围扫掠平面来获取每个像素上的深度信息。所提出的框架显示出适用于各种所需应用的精确密集重建结果。定性和定量评估均表明，与现有方法相比，我们的方法始终能生成更好的深度图。
