Zheng Bolun, Yuan Shanxin, Yan Chenggang, Tian Xiang, Zhang Jiyong, Sun Yaoqi, Liu Lin, Leonardis Ales, Slabaugh Gregory
IEEE Trans Pattern Anal Mach Intell. 2022 Nov;44(11):7705-7717. doi: 10.1109/TPAMI.2021.3115139. Epub 2022 Oct 4.
Image demoireing is a multi-faceted image restoration task involving both moire pattern removal and color restoration. In this paper, we raise a general degradation model to describe an image contaminated by moire patterns, and propose a novel multi-scale bandpass convolutional neural network (MBCNN) for single image demoireing. For moire pattern removal, we propose a multi-block-size learnable bandpass filters (M-LBFs), based on a block-wise frequency domain transform, to learn the frequency domain priors of moire patterns. We also introduce a new loss function named Dilated Advanced Sobel loss (D-ASL) to better sense the frequency information. For color restoration, we propose a two-step tone mapping strategy, which first applies a global tone mapping to correct for a global color shift, and then performs local fine tuning of the color per pixel. To determine the most appropriate frequency domain transform, we investigate several transforms including DCT, DFT, DWT, learnable non-linear transform and learnable orthogonal transform. We finally adopt the DCT. Our basic model won the AIM2019 demoireing challenge. Experimental results on three public datasets show that our method outperforms state-of-the-art methods by a large margin.
图像去网纹是一项多方面的图像恢复任务,涉及去除莫尔条纹和颜色恢复。在本文中,我们提出了一个通用的退化模型来描述受莫尔条纹污染的图像,并提出了一种新颖的多尺度带通卷积神经网络(MBCNN)用于单图像去网纹。对于去除莫尔条纹,我们基于逐块频域变换提出了一种多块大小可学习带通滤波器(M-LBFs),以学习莫尔条纹的频域先验。我们还引入了一种名为扩张高级索贝尔损失(D-ASL)的新损失函数,以更好地感知频率信息。对于颜色恢复,我们提出了一种两步色调映射策略,该策略首先应用全局色调映射来校正全局颜色偏移,然后对每个像素的颜色进行局部微调。为了确定最合适的频域变换,我们研究了几种变换,包括离散余弦变换(DCT)、离散傅里叶变换(DFT)、离散小波变换(DWT)、可学习非线性变换和可学习正交变换。我们最终采用了DCT。我们的基本模型赢得了AIM2019去网纹挑战赛。在三个公共数据集上的实验结果表明,我们的方法在很大程度上优于现有方法。