Luo Jundan, Huang Zhaoyang, Li Yijin, Zhou Xiaowei, Zhang Guofeng, Bao Hujun
IEEE Trans Vis Comput Graph. 2020 Dec;26(12):3434-3445. doi: 10.1109/TVCG.2020.3023565. Epub 2020 Nov 10.
Intrinsic image decomposition, i.e., decomposing a natural image into a reflectance image and a shading image, is used in many augmented reality applications for achieving better visual coherence between virtual contents and real scenes. The main challenge is that the decomposition is ill-posed, especially in indoor scenes where lighting conditions are complicated, while real training data is inadequate. To solve this challenge, we propose NIID-Net, a novel learning-based framework that adapts surface normal knowledge for improving the decomposition. The knowledge learned from relatively more abundant data for surface normal estimation is integrated into intrinsic image decomposition in two novel ways. First, normal feature adapters are proposed to incorporate scene geometry features when decomposing the image. Secondly, a map of integrated lighting is proposed for propagating object contour and planarity information during shading rendering. Furthermore, this map is capable of representing spatially-varying lighting conditions indoors. Experiments show that NIID-Net achieves competitive performance in reflectance estimation and outperforms all previous methods in shading estimation quantitatively and qualitatively. The source code of our implementation is released at https://github.com/zju3dv/NIID-Net.
本征图像分解，即将自然图像分解为反射率图像和阴影图像，在许多增强现实应用中用于在虚拟内容和真实场景之间实现更好的视觉连贯性。主要挑战在于这种分解是不适定的，尤其是在光照条件复杂的室内场景中，而真实的训练数据又不足。为了解决这一挑战，我们提出了NIID-Net，这是一种基于学习的新颖框架，它利用表面法线知识来改进分解。从相对丰富的表面法线估计数据中学到的知识以两种新颖的方式整合到本征图像分解中。首先，提出了法线特征适配器，以便在分解图像时纳入场景几何特征。其次，提出了一个集成光照图，用于在阴影渲染过程中传播物体轮廓和平面信息。此外，该图能够表示室内空间变化的光照条件。实验表明，NIID-Net在反射率估计方面取得了有竞争力的性能，并且在阴影估计方面在定量和定性上均优于所有先前的方法。我们实现的源代码可在https://github.com/zju3dv/NIID-Net上获取。
