Kim Youngjung, Ham Bumsub, Do Minh N, Sohn Kwanghoon
IEEE Trans Image Process. 2018 Dec 24. doi: 10.1109/TIP.2018.2889531.
Most variational formulations for structure-texture image decomposition force structure images to have small norm in some functional spaces, and share a common notion of edges, i.e., large-gradients or -intensity differences. However, such definition makes it difficult to distinguish structure edges from oscillations that have fine spatial scale but high contrast. In this paper, we introduce a new model by learning deep variational prior for structure images without explicit training data. An alternating direction method of multiplier (ADMM) algorithm and its modular structure are adopted to plug deep variational priors into an iterative smoothing process. The central observations are that convolution neural networks (CNNs) can replace the total variation prior, and are indeed powerful to capture the natures of structure and texture. We show that our learned priors using CNNs successfully differentiate highamplitude details from structure edges, and avoid halo artifacts. Different from previous data-driven smoothing schemes, our formulation provides another degree of freedom to produce continuous smoothing effects. Experimental results demonstrate the effectiveness of our approach on various computational photography and image processing applications, including texture removal, detail manipulation, HDR tone-mapping, and nonphotorealistic abstraction.
大多数用于结构 - 纹理图像分解的变分公式迫使结构图像在某些函数空间中具有小范数，并且共享一种共同的边缘概念，即大梯度或强度差异。然而，这样的定义使得难以将结构边缘与具有精细空间尺度但高对比度的振荡区分开来。在本文中，我们通过在没有明确训练数据的情况下学习结构图像的深度变分先验来引入一种新模型。采用交替方向乘子法（ADMM）算法及其模块化结构将深度变分先验插入到迭代平滑过程中。核心观点是卷积神经网络（CNN）可以替代总变分先验，并且确实能够强大地捕捉结构和纹理的本质。我们表明，我们使用CNN学习的先验成功地将高幅度细节与结构边缘区分开来，并避免了光晕伪影。与以前的数据驱动平滑方案不同，我们的公式提供了另一种自由度来产生连续的平滑效果。实验结果证明了我们的方法在各种计算摄影和图像处理应用中的有效性，包括纹理去除、细节处理、HDR色调映射和非真实感抽象。
