Zhang Juyong, Chen Keyu, Zheng Jianmin
IEEE Trans Vis Comput Graph. 2022 Feb;28(2):1274-1287. doi: 10.1109/TVCG.2020.3013876. Epub 2021 Dec 30.
Facial expression retargeting from humans to virtual characters is a useful technique in computer graphics and animation. Traditional methods use markers or blendshapes to construct a mapping between the human and avatar faces. However, these approaches require a tedious 3D modeling process, and the performance relies on the modelers' experience. In this article, we propose a brand-new solution to this cross-domain expression transfer problem via nonlinear expression embedding and expression domain translation. We first build low-dimensional latent spaces for the human and avatar facial expressions with variational autoencoder. Then we construct correspondences between the two latent spaces guided by geometric and perceptual constraints. Specifically, we design geometric correspondences to reflect geometric matching and utilize a triplet data structure to express users' perceptual preference of avatar expressions. A user-friendly method is proposed to automatically generate triplets for a system allowing users to easily and efficiently annotate the correspondences. Using both geometric and perceptual correspondences, we trained a network for expression domain translation from human to avatar. Extensive experimental results and user studies demonstrate that even nonprofessional users can apply our method to generate high-quality facial expression retargeting results with less time and effort.
从人类面部表情重定向到虚拟角色是计算机图形学和动画领域中的一项实用技术。传统方法使用标记或混合形状来构建人类面部与虚拟角色面部之间的映射。然而，这些方法需要繁琐的三维建模过程，并且性能依赖于建模者的经验。在本文中，我们通过非线性表情嵌入和表情域转换，针对这一跨域表情转移问题提出了一种全新的解决方案。我们首先使用变分自编码器为人类和虚拟角色的面部表情构建低维潜在空间。然后，在几何和感知约束的引导下，我们在两个潜在空间之间建立对应关系。具体而言，我们设计几何对应关系以反映几何匹配，并利用三元组数据结构来表达用户对虚拟角色表情的感知偏好。我们提出了一种用户友好的方法，用于为系统自动生成三元组，从而允许用户轻松高效地标注对应关系。利用几何和感知对应关系，我们训练了一个用于从人类到虚拟角色表情域转换的网络。大量实验结果和用户研究表明，即使是非专业用户也可以应用我们的方法，以更少的时间和精力生成高质量的面部表情重定向结果。
