Sariyanidi Evangelos, Zampella Casey J, Schultz Robert T, Tunc Birkan
Center for Autism Research, Children's Hospital of Philadelphia.
University of Pennsylvania.
Comput Vis ECCV. 2020;12354:433-449.
Fitting 3D morphable models (3DMMs) on faces is a well-studied problem, motivated by various industrial and research applications. 3DMMs express a 3D facial shape as a linear sum of basis functions. The resulting shape, however, is a plausible face only when the basis coefficients take values within limited intervals. Methods based on unconstrained optimization address this issue with a weighted penalty on coefficients; however, determining the weight of this penalty is difficult, and the existence of a single weight that works universally is questionable. We propose a new formulation that does not require the tuning of any weight parameter. Specifically, we formulate 3DMM fitting as an inequality-constrained optimization problem, where the primary constraint is that basis coefficients should not exceed the interval that is learned when the 3DMM is constructed. We employ additional constraints to exploit sparse landmark detectors, by forcing the facial shape to be within the error bounds of a reliable detector. To enable operation "in-the-wild", we use a robust objective function, namely Gradient Correlation. Our approach performs comparably with deep learning (DL) methods on "in-the-wild" data that have inexact ground truth, and better than DL methods on more controlled data with exact ground truth. Since our formulation does not require any learning, it enjoys a versatility that allows it to operate with multiple frames of arbitrary sizes. This study's results encourage further research on 3DMM fitting with inequality-constrained optimization methods, which have been unexplored compared to unconstrained methods.
在面部拟合3D可变形模型（3DMM）是一个经过充分研究的问题，受到各种工业和研究应用的推动。3DMM将3D面部形状表示为基函数的线性和。然而，只有当基系数在有限区间内取值时，得到的形状才是一个合理的面部。基于无约束优化的方法通过对系数施加加权惩罚来解决这个问题；然而，确定这种惩罚的权重很困难，而且存在一个普遍适用的单一权重也值得怀疑。我们提出了一种新的公式，该公式不需要调整任何权重参数。具体来说，我们将3DMM拟合公式化为一个不等式约束优化问题，其中主要约束是基系数不应超过在构建3DMM时所学习的区间。我们采用额外的约束来利用稀疏地标检测器，通过强制面部形状在可靠检测器的误差范围内。为了实现“在自然环境中”的操作，我们使用了一个鲁棒的目标函数，即梯度相关性。我们的方法在具有不精确地面真值的“在自然环境中”的数据上与深度学习（DL）方法表现相当，并且在具有精确地面真值的更受控数据上比DL方法表现更好。由于我们的公式不需要任何学习，它具有通用性，允许它在任意大小的多帧上运行。这项研究的结果鼓励进一步研究使用不等式约束优化方法进行3DMM拟合，与无约束方法相比，这些方法尚未得到探索。
