Liu Yun-Fu, Jaw Da-Wei, Huang Shih-Chia, Hwang Jenq-Neng
IEEE Trans Image Process. 2018 Feb 14. doi: 10.1109/TIP.2018.2806202.
Existing learning-based atmospheric particle-removal approaches such as those used for rainy and hazy images are designed with strong assumptions regarding spatial frequency, trajectory, and translucency. However, the removal of snow particles is more complicated because they possess additional attributes of particle size and shape, and these attributes may vary within a single image. Currently, hand-crafted features are still the mainstream for snow removal, making significant generalization difficult to achieve. In response, we have designed a multistage network named DesnowNet to in turn deal with the removal of translucent and opaque snow particles. We also differentiate snow attributes of translucency and chromatic aberration for accurate estimation. Moreover, our approach individually estimates residual complements of the snow-free images to recover details obscured by opaque snow. Additionally, a multi-scale design is utilized throughout the entire network to model the diversity of snow. As demonstrated in the qualitative and quantitative experiments, our approach outperforms state-of-the-art learning-based atmospheric phenomena removal methods and one semantic segmentation baseline on the proposed Snow100K dataset. The results indicate our network would benefit applications involving computer vision and graphics.
现有的基于学习的大气颗粒去除方法,如用于处理雨天和模糊图像的方法,是在对空间频率、轨迹和半透明度做出强假设的情况下设计的。然而,去除雪颗粒更为复杂,因为它们具有颗粒大小和形状等附加属性,并且这些属性可能在单个图像中有所不同。目前,手工特征仍然是除雪的主流方法,难以实现显著的泛化。对此,我们设计了一个名为DesnowNet的多阶段网络,依次处理半透明和不透明雪颗粒的去除。我们还区分了半透明度和色差的雪属性,以进行准确估计。此外,我们的方法单独估计无雪图像的残余补片,以恢复被不透明雪遮挡的细节。此外,在整个网络中采用多尺度设计来对雪的多样性进行建模。如定性和定量实验所示,我们的方法在所提出的Snow100K数据集上优于基于学习的最先进的大气现象去除方法和一个语义分割基线。结果表明,我们的网络将有益于涉及计算机视觉和图形学的应用。