IEEE Trans Image Process. 2021;30:6024-6035. doi: 10.1109/TIP.2021.3090658. Epub 2021 Jul 1.
Existing GAN-based multi-view face synthesis methods rely heavily on "creating" faces, and thus they struggle in reproducing the faithful facial texture and fail to preserve identity when undergoing a large angle rotation. In this paper, we combat this problem by dividing the challenging large-angle face synthesis into a series of easy small-angle rotations, and each of them is guided by a face flow to maintain faithful facial details. In particular, we propose a Face Flow-guided Generative Adversarial Network (FFlowGAN) that is specifically trained for small-angle synthesis. The proposed network consists of two modules, a face flow module that aims to compute a dense correspondence between the input and target faces. It provides strong guidance to the second module, face synthesis module, for emphasizing salient facial texture. We apply FFlowGAN multiple times to progressively synthesize different views, and therefore facial features can be propagated to the target view from the very beginning. All these multiple executions are cascaded and trained end-to-end with a unified back-propagation, and thus we ensure each intermediate step contributes to the final result. Extensive experiments demonstrate the proposed divide-and-conquer strategy is effective, and our method outperforms the state-of-the-art on four benchmark datasets qualitatively and quantitatively.
现有的基于 GAN 的多视角人脸合成方法严重依赖于“生成”人脸，因此在复制真实的面部纹理时存在困难，并且在进行大角度旋转时无法保留身份。在本文中，我们通过将具有挑战性的大角度人脸合成问题分解为一系列容易的小角度旋转问题来解决这个问题，并且每个旋转都由人脸流引导，以保持真实的面部细节。特别是，我们提出了一种专门用于小角度合成的人脸流引导生成对抗网络（FFlowGAN）。所提出的网络由两个模块组成，即人脸流模块，旨在计算输入和目标人脸之间的密集对应关系。它为第二个模块——人脸合成模块提供了强大的指导，以强调显著的面部纹理。我们多次应用 FFlowGAN 来逐步合成不同的视角，因此可以从一开始就将面部特征从源视角传播到目标视角。所有这些多次执行都是级联的，并通过统一的反向传播进行端到端训练，因此我们确保每个中间步骤都对最终结果有贡献。广泛的实验证明，所提出的分而治之策略是有效的，我们的方法在四个基准数据集上在定性和定量方面都优于最先进的方法。
