Zhang Qing, Zhou Jin, Zhu Lei, Sun Wei, Xiao Chunxia, Zheng Wei-Shi
IEEE Trans Pattern Anal Mach Intell. 2022 Dec;44(12):9669-9686. doi: 10.1109/TPAMI.2021.3129795. Epub 2022 Nov 7.
Recent learning-based intrinsic image decomposition methods have achieved remarkable progress. However, they usually require massive ground truth intrinsic images for supervised learning, which limits their applicability on real-world images since obtaining ground truth intrinsic decomposition for natural images is very challenging. In this paper, we present an unsupervised framework that is able to learn the decomposition effectively from a single natural image by training solely with the image itself. Our approach is built upon the observations that the reflectance of a natural image typically has high internal self-similarity of patches, and a convolutional generation network tends to boost the self-similarity of an image when trained for image reconstruction. Based on the observations, an unsupervised intrinsic decomposition network (UIDNet) consisting of two fully convolutional encoder-decoder sub-networks, i.e., reflectance prediction network (RPN) and shading prediction network (SPN), is devised to decompose an image into reflectance and shading by promoting the internal self-similarity of the reflectance component, in a way that jointly trains RPN and SPN to reproduce the given image. A novel loss function is also designed to make effective the training for intrinsic decomposition. Experimental results on three benchmark real-world datasets demonstrate the superiority of the proposed method.
最近基于学习的固有图像分解方法取得了显著进展。然而，它们通常需要大量的真实固有图像用于监督学习，这限制了它们在真实世界图像上的适用性，因为获取自然图像的真实固有分解非常具有挑战性。在本文中，我们提出了一个无监督框架，该框架能够仅通过使用图像本身进行训练，从单个自然图像中有效地学习分解。我们的方法基于以下观察结果：自然图像的反射率通常具有较高的块内自相似性，并且卷积生成网络在训练用于图像重建时倾向于增强图像的自相似性。基于这些观察结果，设计了一个由两个全卷积编码器 - 解码器子网络组成的无监督固有分解网络（UIDNet），即反射率预测网络（RPN）和阴影预测网络（SPN），通过促进反射率分量的内部自相似性将图像分解为反射率和阴影，以联合训练RPN和SPN来重现给定图像的方式。还设计了一种新颖的损失函数以使固有分解的训练有效。在三个基准真实世界数据集上的实验结果证明了所提出方法的优越性。
