IEEE Trans Cybern. 2020 Jun;50(6):2701-2714. doi: 10.1109/TCYB.2019.2924589. Epub 2019 Jul 16.
Face sketch synthesis is a crucial technique in digital entertainment. However, the existing face sketch synthesis approaches usually generate face sketches with coarse structures. The fine details on some facial components fail to be generated. In this paper, inspired by the artists during drawing face sketches, we propose a bionic face sketch generator. It includes three parts: 1) a coarse part; 2) a fine part; and 3) a finer part. The coarse part builds the facial structure of a sketch by a generative adversarial network in the U-Net. In the middle part, the noise produced by the coarse part is erased and the fine details on the important face components are generated via a probabilistic graphic model. To compensate for the fine sketch with distinctive edge and area of shadows and lights, we learn a mapping relationship at the high-frequency band by a convolutional neural network in the finer part. The experimental results show that the proposed bionic face sketch generator can synthesize the face sketch with more delicate and striking details, satisfy the requirement of users in the digital entertainment, and provide the students with the coarse, fine, and finer face sketch copies when learning sketches. Compared with the state-of-the-art methods, the proposed approach achieves better results in both visual effects and quantitative metrics.
人脸素描合成是数字娱乐中的一项关键技术。然而，现有的人脸素描合成方法通常生成的人脸素描结构较为粗糙，一些面部组件的精细细节无法生成。受绘画人脸素描艺术家的启发，我们提出了一种仿生人脸素描生成器。它包括三个部分：1）粗糙部分；2）精细部分；3）更精细部分。粗糙部分通过 U-Net 中的生成对抗网络构建素描的面部结构。在中间部分，通过概率图形模型擦除粗糙部分产生的噪声，并生成重要面部组件上的精细细节。为了补偿具有独特边缘和光影区域的精细素描，我们在更精细的部分通过卷积神经网络学习高频带的映射关系。实验结果表明，所提出的仿生人脸素描生成器可以合成具有更精细、更引人注目的细节的人脸素描，满足数字娱乐用户的需求，并为学习素描的学生提供粗糙、精细和更精细的人脸素描副本。与现有的方法相比，该方法在视觉效果和定量指标上都取得了更好的效果。
