Jia Kui, Gong Shaogang
Shenzhen Institute of Advanced Integration Technology, Chinese Academy of Sciences/Chinese Academy of Hong Kong, Shenzhen, China.
IEEE Trans Image Process. 2008 Jun;17(6):873-86. doi: 10.1109/TIP.2008.922421.
Existing learning-based face super-resolution (hallucination) techniques generate high-resolution images of a single facial modality (i.e., at a fixed expression, pose and illumination) given one or set of low-resolution face images as probe. Here, we present a generalized approach based on a hierarchical tensor (multilinear) space representation for hallucinating high-resolution face images across multiple modalities, achieving generalization to variations in expression and pose. In particular, we formulate a unified tensor which can be reduced to two parts: a global image-based tensor for modeling the mappings among different facial modalities, and a local patch-based multiresolution tensor for incorporating high-resolution image details. For realistic hallucination of unregistered low-resolution faces contained in raw images, we develop an automatic face alignment algorithm capable of pixel-wise alignment by iteratively warping the probing face to its projection in the space of training face images. Our experiments show not only performance superiority over existing benchmark face super-resolution techniques on single modal face hallucination, but also novelty of our approach in coping with multimodal hallucination and its robustness in automatic alignment under practical imaging conditions.
现有的基于学习的人脸超分辨率（幻觉）技术，在给定一幅或一组低分辨率人脸图像作为输入的情况下，生成单一人脸模态（即固定表情、姿态和光照条件下）的高分辨率图像。在此，我们提出一种基于分层张量（多线性）空间表示的通用方法，用于跨多种模态生成高分辨率人脸图像，实现对表情和姿态变化的泛化。具体而言，我们构建了一个统一的张量，它可分解为两部分：一个基于全局图像的张量，用于对不同人脸模态之间的映射进行建模；一个基于局部块的多分辨率张量，用于融入高分辨率图像细节。为了对原始图像中未配准的低分辨率人脸进行逼真的幻觉生成，我们开发了一种自动人脸对齐算法，该算法通过将探测人脸迭代扭曲到其在训练人脸图像空间中的投影，实现逐像素对齐。我们的实验表明，该方法不仅在单模态人脸幻觉生成方面优于现有的基准人脸超分辨率技术，而且在处理多模态幻觉生成时具有创新性，在实际成像条件下的自动对齐中具有鲁棒性。
