Zhao Ruoyu, Zhu Mingrui, Wang Nannan, Gao Xinbo
IEEE Trans Neural Netw Learn Syst. 2025 Mar;36(3):4492-4503. doi: 10.1109/TNNLS.2024.3377609. Epub 2025 Feb 28.
Face stylization has made notable progress in recent years. However, when training on limited data, the performance of existing approaches significantly declines. Although some studies have attempted to tackle this problem, they either failed to achieve the few-shot setting (less than 10) or can only get suboptimal results. In this article, we propose GAN Prior Distillation (GPD) to enable effective few-shot face stylization. GPD contains two models: a teacher network with GAN Prior and a student network that fulfills end-to-end translation. Specifically, we adapt the teacher network trained on large-scale data in the source domain to the target domain using a handful of samples, where it can learn the target domain's knowledge. Then, we can achieve few-shot augmentation by generating source domain and target domain images simultaneously with the same latent codes. We propose an anchor-based knowledge distillation module that can fully use the difference between the training and the augmented data to distill the knowledge of the teacher network into the student network. The trained student network achieves excellent generalization performance with the absorption of additional knowledge. Qualitative and quantitative experiments demonstrate that our method achieves superior results than state-of-the-art approaches in a few-shot setting.
近年来,面部风格化取得了显著进展。然而,在有限数据上进行训练时,现有方法的性能会显著下降。尽管一些研究试图解决这个问题,但它们要么未能实现少样本设置(少于10个样本),要么只能得到次优结果。在本文中,我们提出了生成对抗网络先验蒸馏(GAN Prior Distillation,GPD)方法,以实现有效的少样本面部风格化。GPD包含两个模型:一个具有GAN先验的教师网络和一个实现端到端翻译的学生网络。具体来说,我们使用少量样本将在源域大规模数据上训练的教师网络适配到目标域,使其能够学习目标域的知识。然后,我们可以通过使用相同的潜在代码同时生成源域和目标域图像来实现少样本增强。我们提出了一个基于锚点的知识蒸馏模块,该模块可以充分利用训练数据和增强数据之间的差异,将教师网络的知识蒸馏到学生网络中。经过训练的学生网络通过吸收额外知识实现了出色的泛化性能。定性和定量实验表明,在少样本设置下,我们的方法比现有方法取得了更好的结果。