IEEE Trans Pattern Anal Mach Intell. 2020 Nov;42(11):2795-2808. doi: 10.1109/TPAMI.2019.2919616. Epub 2019 May 28.
Shadow detection and shadow removal are fundamental and challenging tasks, requiring an understanding of the global image semantics. This paper presents a novel deep neural network design for shadow detection and removal by analyzing the spatial image context in a direction-aware manner. To achieve this, we first formulate the direction-aware attention mechanism in a spatial recurrent neural network (RNN) by introducing attention weights when aggregating spatial context features in the RNN. By learning these weights through training, we can recover direction-aware spatial context (DSC) for detecting and removing shadows. This design is developed into the DSC module and embedded in a convolutional neural network (CNN) to learn the DSC features at different levels. Moreover, we design a weighted cross entropy loss to make effective the training for shadow detection and further adopt the network for shadow removal by using a euclidean loss function and formulating a color transfer function to address the color and luminosity inconsistencies in the training pairs. We employed two shadow detection benchmark datasets and two shadow removal benchmark datasets, and performed various experiments to evaluate our method. Experimental results show that our method performs favorably against the state-of-the-art methods for both shadow detection and shadow removal.
阴影检测和去除是基本且具有挑战性的任务，需要理解全局图像语义。本文提出了一种新的深度神经网络设计，通过以方向感知的方式分析空间图像上下文来进行阴影检测和去除。为此，我们首先在空间递归神经网络（RNN）中通过在 RNN 中聚合空间上下文特征时引入注意力权重来制定方向感知注意力机制。通过训练学习这些权重，我们可以恢复用于检测和去除阴影的方向感知空间上下文（DSC）。这个设计被开发成 DSC 模块，并嵌入到卷积神经网络（CNN）中，以学习不同层次的 DSC 特征。此外，我们设计了加权交叉熵损失函数来有效地进行阴影检测训练，并进一步采用该网络进行阴影去除，通过使用欧几里得损失函数和制定颜色传递函数来解决训练对中的颜色和亮度不一致性问题。我们使用了两个阴影检测基准数据集和两个阴影去除基准数据集，并进行了各种实验来评估我们的方法。实验结果表明，我们的方法在阴影检测和去除方面都优于最先进的方法。
