IEEE Trans Pattern Anal Mach Intell. 2017 Nov;39(11):2127-2141. doi: 10.1109/TPAMI.2016.2636829. Epub 2016 Dec 7.
Given a photo collection of "unconstrained" face images of one individual captured under a variety of unknown pose, expression, and illumination conditions, this paper presents a method for reconstructing a 3D face surface model of the individual along with albedo information. Unlike prior work on face reconstruction that requires large photo collections, we formulate an approach to adapt to photo collections with a high diversity in both the number of images and the image quality. To achieve this, we incorporate prior knowledge about face shape by fitting a 3D morphable model to form a personalized template, following by using a novel photometric stereo formulation to complete the fine details, under a coarse-to-fine scheme. Our scheme incorporates a structural similarity-based local selection step to help identify a common expression for reconstruction while discarding occluded portions of faces. The evaluation of reconstruction performance is through a novel quality measure, in the absence of ground truth 3D scans. Superior large-scale experimental results are reported on synthetic, Internet, and personal photo collections.
给定一个人的“无约束”面部图像的照片集合，这些图像是在各种未知的姿势、表情和光照条件下拍摄的，本文提出了一种方法来重建该个体的 3D 面部表面模型以及反射率信息。与需要大量照片集合的先前的面部重建工作不同，我们提出了一种方法来适应具有高度多样性的照片集合，无论是在图像数量还是图像质量方面。为了实现这一点，我们通过拟合 3D 可变形模型来形成个性化模板，结合新颖的光度立体公式来完成精细的细节，采用粗到细的方案。我们的方案结合了基于结构相似性的局部选择步骤，有助于在排除面部遮挡部分的同时，为重建找到一个共同的表情。在没有地面真实 3D 扫描的情况下，通过一种新颖的质量度量来评估重建性能。在合成、互联网和个人照片集合上进行了优越的大规模实验。
