Le Hieu, Samaras Dimitris
IEEE Trans Pattern Anal Mach Intell. 2022 Dec;44(12):9088-9101. doi: 10.1109/TPAMI.2021.3124934. Epub 2022 Nov 7.
We propose a novel deep learning method for shadow removal. Inspired by physical models of shadow formation, we use a linear illumination transformation to model the shadow effects in the image that allows the shadow image to be expressed as a combination of the shadow-free image, the shadow parameters, and a matte layer. We use two deep networks, namely SP-Net and M-Net, to predict the shadow parameters and the shadow matte respectively. This system allows us to remove the shadow effects from images. We then employ an inpainting network, I-Net, to further refine the results. We train and test our framework on the most challenging shadow removal dataset (ISTD). Our method improves the state-of-the-art in terms of mean absolute error (MAE) for the shadow area by 20%. Furthermore, this decomposition allows us to formulate a patch-based weakly-supervised shadow removal method. This model can be trained without any shadow- free images (that are cumbersome to acquire) and achieves competitive shadow removal results compared to state-of-the-art methods that are trained with fully paired shadow and shadow-free images. Last, we introduce SBU-Timelapse, a video shadow removal dataset for evaluating shadow removal methods.
我们提出了一种用于阴影去除的新型深度学习方法。受阴影形成物理模型的启发，我们使用线性光照变换对图像中的阴影效果进行建模，使得阴影图像可以表示为无阴影图像、阴影参数和蒙版层的组合。我们使用两个深度网络，即SP-Net和M-Net，分别预测阴影参数和阴影蒙版。该系统使我们能够从图像中去除阴影效果。然后，我们使用修复网络I-Net进一步优化结果。我们在最具挑战性的阴影去除数据集（ISTD）上对我们的框架进行训练和测试。我们的方法在阴影区域的平均绝对误差（MAE）方面比当前最先进的方法提高了20%。此外，这种分解使我们能够制定一种基于补丁的弱监督阴影去除方法。该模型无需任何无阴影图像（获取起来很麻烦）即可训练，并且与使用完全配对的阴影和无阴影图像训练的当前最先进方法相比，在阴影去除方面取得了具有竞争力的结果。最后，我们引入了SBU-Timelapse，一个用于评估阴影去除方法的视频阴影去除数据集。
