Long Xiaoxiao, Zheng Yuhang, Zheng Yupeng, Tian Beiwen, Lin Cheng, Liu Lingjie, Zhao Hao, Zhou Guyue, Wang Wenping
IEEE Trans Pattern Anal Mach Intell. 2024 Sep;46(9):6263-6279. doi: 10.1109/TPAMI.2024.3381710. Epub 2024 Aug 6.
We introduce a novel approach to learn geometries such as depth and surface normal from images while incorporating geometric context. The difficulty of reliably capturing geometric context in existing methods impedes their ability to accurately enforce the consistency between the different geometric properties, thereby leading to a bottleneck of geometric estimation quality. We therefore propose the Adaptive Surface Normal (ASN) constraint, a simple yet efficient method. Our approach extracts geometric context that encodes the geometric variations present in the input image and correlates depth estimation with geometric constraints. By dynamically determining reliable local geometry from randomly sampled candidates, we establish a surface normal constraint, where the validity of these candidates is evaluated using the geometric context. Furthermore, our normal estimation leverages the geometric context to prioritize regions that exhibit significant geometric variations, which makes the predicted normals accurately capture intricate and detailed geometric information. Through the integration of geometric context, our method unifies depth and surface normal estimations within a cohesive framework, which enables the generation of high-quality 3D geometry from images. We validate the superiority of our approach over state-of-the-art methods through extensive evaluations and comparisons on diverse indoor and outdoor datasets, showcasing its efficiency and robustness.
我们引入了一种新颖的方法,可在纳入几何上下文的同时从图像中学习诸如深度和表面法线等几何信息。现有方法中可靠捕捉几何上下文的困难阻碍了它们准确强制不同几何属性之间一致性的能力,从而导致几何估计质量的瓶颈。因此,我们提出了自适应表面法线(ASN)约束,这是一种简单而有效的方法。我们的方法提取对输入图像中存在的几何变化进行编码的几何上下文,并将深度估计与几何约束相关联。通过从随机采样的候选中动态确定可靠的局部几何,我们建立了一个表面法线约束,其中使用几何上下文评估这些候选的有效性。此外,我们的法线估计利用几何上下文对呈现显著几何变化的区域进行优先级排序,这使得预测的法线能够准确捕捉复杂而详细的几何信息。通过整合几何上下文,我们的方法在一个连贯的框架内统一了深度和表面法线估计,从而能够从图像生成高质量的3D几何信息。我们通过在各种室内和室外数据集上进行广泛的评估和比较,验证了我们的方法相对于现有方法的优越性,展示了其效率和鲁棒性。