Ito Shintaro, Miura Kanta, Ito Koichi, Aoki Takafumi
Graduate School of Information Sciences, Tohoku University, 6-6-05, Aramaki Aza Aoba, Sendai 9808579, Japan.
J Imaging. 2024 Mar 8;10(3):68. doi: 10.3390/jimaging10030068.
In this paper, we propose a method to refine the depth maps obtained by Multi-View Stereo (MVS) through iterative optimization of the Neural Radiance Field (NeRF). MVS accurately estimates the depths on object surfaces, and NeRF accurately estimates the depths at object boundaries. The key ideas of the proposed method are to combine MVS and NeRF to utilize the advantages of both in depth map estimation and to use NeRF for depth map refinement. We also introduce a Huber loss into the NeRF optimization to improve the accuracy of the depth map refinement, where the Huber loss reduces the estimation error in the radiance fields by placing constraints on errors larger than a threshold. Through a set of experiments using the Redwood-3dscan dataset and the DTU dataset, which are public datasets consisting of multi-view images, we demonstrate the effectiveness of the proposed method compared to conventional methods: COLMAP, NeRF, and DS-NeRF.
在本文中，我们提出了一种通过对神经辐射场（NeRF）进行迭代优化来细化多视图立体视觉（MVS）获得的深度图的方法。MVS能准确估计物体表面的深度，而NeRF能准确估计物体边界处的深度。该方法的关键思想是将MVS和NeRF相结合，以利用两者在深度图估计方面的优势，并使用NeRF进行深度图细化。我们还在NeRF优化中引入了Huber损失，以提高深度图细化的准确性，其中Huber损失通过对大于阈值的误差施加约束来减少辐射场中的估计误差。通过使用Redwood - 3dscan数据集和DTU数据集进行的一组实验（这两个数据集是由多视图图像组成的公共数据集），我们证明了与传统方法（COLMAP、NeRF和DS - NeRF）相比，该方法的有效性。
