Hu Zhongyun, Nsampi Ntumba Elie, Wang Xue, Wang Qing
IEEE Trans Image Process. 2022;31:3935-3948. doi: 10.1109/TIP.2022.3177311. Epub 2022 Jun 9.
Existing any-to-any relighting methods suffer from the task-aliasing effects and the loss of local details in the image generation process, such as shading and attached-shadow. In this paper, we present PNRNet, a novel neural architecture that decomposes the any-to-any relighting task into three simpler sub-tasks, i.e. lighting estimation, color temperature transfer, and lighting direction transfer, to avoid the task-aliasing effects. These sub-tasks are easy to learn and can be trained with direct supervisions independently. To better preserve local shading and attached-shadow details, we propose a parallel multi-scale network that incorporates multiple physical attributes to model local illuminations for lighting direction transfer. We also introduce a simple yet effective color temperature transfer network to learn a pixel-level non-linear function which allows color temperature adjustment beyond the predefined color temperatures and generalizes well to real images. Extensive experiments demonstrate that our proposed approach achieves better results quantitatively and qualitatively than prior works.
现有的任意到任意的重光照方法在图像生成过程中存在任务混叠效应和局部细节丢失的问题，例如阴影和附着阴影。在本文中，我们提出了PNRNet，这是一种新颖的神经架构，它将任意到任意的重光照任务分解为三个更简单的子任务，即光照估计、色温转换和光照方向转换，以避免任务混叠效应。这些子任务易于学习，并且可以独立地通过直接监督进行训练。为了更好地保留局部阴影和附着阴影细节，我们提出了一种并行多尺度网络，该网络结合了多个物理属性来为光照方向转换建模局部光照。我们还引入了一个简单而有效的色温转换网络，以学习像素级非线性函数，该函数允许在预定义的色温之外进行色温调整，并且能很好地推广到真实图像。大量实验表明，我们提出的方法在定量和定性方面都比先前的工作取得了更好的结果。
