IEEE Trans Image Process. 2017 Dec;26(12):5966-5979. doi: 10.1109/TIP.2017.2740624. Epub 2017 Aug 16.
We present a novel templateless approach for nonrigid reconstruction and motion tracking using a single RGB-D camera. Without any template prior, our system achieves accurate reconstruction and tracking for considerably deformable objects. To robustly register the input sequence of partial depth scans with dynamic motion, we propose an efficient local-to-global hierarchical optimization framework inspired by the idea of traditional structure-from-motion. Our proposed framework mainly consists of two stages, local nonrigid bundle adjustment and global optimization. To eliminate error accumulation during the nonrigid registration of loop motion sequences, we split the full sequence into several segments and apply local nonrigid bundle adjustment to align each segment locally. Global optimization is then adopted to combine all segments and handle the drift problem through loop-closure constraint. By fitting to the input partial data, a deforming 3D model sequence of dynamic objects is finally generated. Experiments on both synthetic and real test data sets and comparisons with state of the art demonstrate that our approach can handle considerable motions robustly and efficiently, and reconstruct high-quality 3D model sequences without drift.
我们提出了一种新颖的无模板方法，用于使用单个 RGB-D 相机进行非刚性重建和运动跟踪。在没有任何模板先验的情况下，我们的系统能够实现对相当大变形物体的精确重建和跟踪。为了稳健地将具有动态运动的部分深度扫描输入序列与动态运动进行配准，我们提出了一种受传统运动恢复结构思想启发的高效局部到全局分层优化框架。我们提出的框架主要包括两个阶段，局部非刚性捆绑调整和全局优化。为了消除环运动序列非刚性配准过程中的误差积累，我们将整个序列分成几个片段，并应用局部非刚性捆绑调整来局部对齐每个片段。然后采用全局优化将所有片段组合在一起，并通过闭环约束处理漂移问题。通过拟合输入的部分数据，最终生成动态对象的变形 3D 模型序列。在合成和真实测试数据集上的实验以及与现有技术的比较表明，我们的方法能够稳健高效地处理相当大的运动，并重建无漂移的高质量 3D 模型序列。
