IEEE Trans Image Process. 2023;32:3455-3464. doi: 10.1109/TIP.2023.3285439. Epub 2023 Jun 23.
We focus on addressing the problem of shadow removal for an image, and attempt to make a weakly supervised learning model that does not depend on the pixelwise-paired training samples, but only uses the samples with image-level labels that indicate whether an image contains shadow or not. To this end, we propose a deep reciprocal learning model that interactively optimizes the shadow remover and the shadow detector to improve the overall capability of the model. On the one hand, shadow removal is modeled as an optimization problem with a latent variable of the detected shadow mask. On the other hand, a shadow detector can be trained using the prior from the shadow remover. A self-paced learning strategy is employed to avoid fitting to intermediate noisy annotation during the interactive optimization. Furthermore, a color-maintenance loss and a shadow-attention discriminator are both designed to facilitate model optimization. Extensive experiments on the pairwise ISTD dataset, SRD dataset, and unpaired USR dataset demonstrate the superiority of the proposed deep reciprocal model.
我们专注于解决图像去影问题，并尝试构建一个无需像素级配对训练样本的弱监督学习模型，而仅使用具有图像级标签的样本，这些标签指示图像是否包含阴影。为此，我们提出了一种深度互学习模型，该模型可以交互优化去影器和阴影检测器，以提高模型的整体性能。一方面，去影过程被建模为一个具有检测到的阴影掩模的潜在变量的优化问题。另一方面，可以使用去影器的先验知识来训练阴影检测器。采用自步学习策略来避免在交互优化过程中拟合中间嘈杂的注释。此外，还设计了颜色保持损失和阴影注意鉴别器，以促进模型优化。在成对的 ISTD 数据集、SRD 数据集和非配对的 USR 数据集上进行的广泛实验表明，所提出的深度互学习模型具有优越性。
