Dong Yongsheng, Tan Wei, Tao Dacheng, Zheng Lintao, Li Xuelong
IEEE Trans Image Process. 2022;31:485-498. doi: 10.1109/TIP.2021.3130539. Epub 2021 Dec 15.
Cartoonization as a special type of artistic style transfer is a difficult image processing task. The current existing artistic style transfer methods cannot generate satisfactory cartoon-style images due to that artistic style images often have delicate strokes and rich hierarchical color changes while cartoon-style images have smooth surfaces without obvious color changes, and sharp edges. To this end, we propose a cartoon loss based generative adversarial network (CartoonLossGAN) for cartoonization. Particularly, we first reuse the encoder part of the discriminator to build a compact generative adversarial network (GAN) based cartoonization architecture. Then we propose a novel cartoon loss function for the architecture. It can imitate the process of sketching to learn the smooth surface of the cartoon image, and imitate the coloring process to learn the coloring of the cartoon image. Furthermore, we also propose an initialization strategy, which is used in the scenario of reusing the discriminator to make our model training easier and more stable. Extensive experimental results demonstrate that our proposed CartoonLossGAN can generate fantastic cartoon-style images, and outperforms four representative methods.
卡通化作为一种特殊类型的艺术风格迁移,是一项具有挑战性的图像处理任务。由于艺术风格图像通常具有细腻的笔触和丰富的层次颜色变化,而卡通风格图像具有光滑的表面、无明显颜色变化以及锐利的边缘,当前现有的艺术风格迁移方法无法生成令人满意的卡通风格图像。为此,我们提出了一种基于卡通损失的生成对抗网络(CartoonLossGAN)用于卡通化。具体而言,我们首先复用判别器的编码器部分来构建一个基于紧凑生成对抗网络(GAN)的卡通化架构。然后我们为该架构提出了一种新颖的卡通损失函数。它可以模仿草图绘制过程来学习卡通图像的光滑表面,并模仿上色过程来学习卡通图像的上色。此外,我们还提出了一种初始化策略,用于在复用判别器的场景中使我们的模型训练更轻松、更稳定。大量实验结果表明,我们提出的CartoonLossGAN可以生成出色的卡通风格图像,并且优于四种代表性方法。