IEEE Trans Image Process. 2019 Dec;28(12):6225-6236. doi: 10.1109/TIP.2019.2924554. Epub 2019 Jun 28.
Though generative adversarial networks (GANs) can hallucinate high-quality high-resolution (HR) faces from low-resolution (LR) faces, they cannot ensure identity preservation during face hallucination, making the HR faces difficult to recognize. To address this problem, we propose a Siamese GAN (SiGAN) to reconstruct HR faces that visually resemble their corresponding identities. On top of a Siamese network, the proposed SiGAN consists of a pair of two identical generators and one discriminator. We incorporate reconstruction error and identity label information in the loss function of SiGAN in a pairwise manner. By iteratively optimizing the loss functions of the generator pair and the discriminator of SiGAN, we not only achieve visually-pleasing face reconstruction but also ensure that the reconstructed information is useful for identity recognition. Experimental results demonstrate that SiGAN significantly outperforms existing face hallucination GANs in objective face verification performance while achieving promising visual-quality reconstruction. Moreover, for input LR faces with unseen identities that are not part of the training dataset, SiGAN can still achieve reasonable performance.
尽管生成对抗网络（GAN）可以从低分辨率（LR）人脸图像中生成高质量的高分辨率（HR）人脸图像，但它们不能在人脸图像生成过程中保证身份信息的保留，这使得生成的 HR 人脸图像难以识别。针对这个问题，我们提出了一种孪生生成对抗网络（SiGAN）来重建与相应身份具有相似视觉特征的 HR 人脸图像。在孪生网络的基础上，我们提出的 SiGAN 由一对两个相同的生成器和一个鉴别器组成。我们在 SiGAN 的损失函数中以成对的方式引入了重建误差和身份标签信息。通过迭代优化生成器对和 SiGAN 鉴别器的损失函数，我们不仅实现了令人愉悦的人脸重建，还保证了重建信息对身份识别的有用性。实验结果表明，SiGAN 在客观的人脸验证性能方面明显优于现有的人脸图像生成 GAN，同时实现了有吸引力的视觉质量重建。此外，对于输入的、来自训练数据集之外的、不为人知的 LR 人脸图像，SiGAN 仍然可以实现合理的性能。
