Huo Jing, Liu Xiangde, Li Wenbin, Gao Yang, Yin Hujun, Luo Jiebo
IEEE Trans Image Process. 2022;31:3347-3358. doi: 10.1109/TIP.2022.3154238. Epub 2022 May 9.
Given a photo of a subject, ability to generate a caricature image that captures distinct characteristics of the subject but with certain exaggeration of their prominent features is of fundamental importance to image processing and facial recognition. There are two main challenges in this task: shape exaggeration and style transfer. The former morphs and exaggerates key facial features of the subject, while the latter generates caricature images in a certain artistic style. In this paper, we propose a CAricature Style Transfer (CAST) framework for caricature generation. There are two modules in the proposed framework. The first is a geometric warping module. Different from the existing style transfer methods, we incorporate the Whitening and Coloring Transformation (WCT) in the geometric style transfer. The WCT is learned on photo and caricature landmarks or the caricature landmark space of a specific artist and is capable of transforming input photo landmarks to caricature landmarks. The second module is a texture style rendering module. We propose a new style transfer method by considering a semantic region-aligned style transfer via affinity constraint. Given a reference caricature image as the style reference, this module is capable of transferring styles between the same or similar semantic regions in caricatures and photos. Furthermore, it can transfer visual attributes of the reference caricatures (such as mouth shape and expressions) to the output caricatures. Experiments have shown desirable effects of the proposed method in transferring both the geometric and artistic texture styles of caricatures. Both qualitative and quantitative results show that the CAST framework is more effective compared than the state-of-the-art caricature generation methods.
给定一张人物照片，生成一张能捕捉人物鲜明特征但对其突出特征进行一定夸张的漫画图像的能力，对于图像处理和面部识别至关重要。这项任务存在两个主要挑战：形状夸张和风格迁移。前者对人物的关键面部特征进行变形和夸张，而后者以某种艺术风格生成漫画图像。在本文中，我们提出了一种用于漫画生成的漫画风格迁移（CAST）框架。所提出的框架中有两个模块。第一个是几何变形模块。与现有的风格迁移方法不同，我们在几何风格迁移中纳入了白化和着色变换（WCT）。WCT是在照片和漫画地标或特定艺术家的漫画地标空间上学习得到的，能够将输入照片地标转换为漫画地标。第二个模块是纹理风格渲染模块。我们通过考虑基于亲和度约束的语义区域对齐风格迁移，提出了一种新的风格迁移方法。给定一张参考漫画图像作为风格参考，该模块能够在漫画和照片的相同或相似语义区域之间迁移风格。此外，它还能将参考漫画的视觉属性（如嘴型和表情）迁移到输出漫画中。实验表明，所提出的方法在迁移漫画的几何和艺术纹理风格方面都有理想的效果。定性和定量结果均表明，CAST框架比当前最先进的漫画生成方法更有效。
