School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA.
IEEE Trans Pattern Anal Mach Intell. 2011 Jun;33(6):1098-115. doi: 10.1109/TPAMI.2010.162.
In this work, we present a nonrigid approach to jointly solving the tasks of 2D-3D pose estimation and 2D image segmentation. In general, most frameworks that couple both pose estimation and segmentation assume that one has exact knowledge of the 3D object. However, under nonideal conditions, this assumption may be violated if only a general class to which a given shape belongs is given (e.g., cars, boats, or planes). Thus, we propose to solve the 2D-3D pose estimation and 2D image segmentation via nonlinear manifold learning of 3D embedded shapes for a general class of objects or deformations for which one may not be able to associate a skeleton model. Thus, the novelty of our method is threefold: first, we present and derive a gradient flow for the task of nonrigid pose estimation and segmentation. Second, due to the possible nonlinear structures of one's training set, we evolve the pre-image obtained through kernel PCA for the task of shape analysis. Third, we show that the derivation for shape weights is general. This allows us to use various kernels, as well as other statistical learning methodologies, with only minimal changes needing to be made to the overall shape evolution scheme. In contrast with other techniques, we approach the nonrigid problem, which is an infinite-dimensional task, with a finite-dimensional optimization scheme. More importantly, we do not explicitly need to know the interaction between various shapes such as that needed for skeleton models as this is done implicitly through shape learning. We provide experimental results on several challenging pose estimation and segmentation scenarios.
在这项工作中，我们提出了一种非刚性方法，用于联合解决二维到三维姿态估计和二维图像分割任务。通常，将姿态估计和分割耦合的大多数框架都假设对三维物体具有准确的了解。然而，在不理想的条件下，如果只给定了给定形状所属的一般类别（例如汽车、船只或飞机），则可能违反此假设。因此，我们建议通过对一般类别的对象或变形进行三维嵌入形状的非线性流形学习来解决二维到三维姿态估计和二维图像分割问题，对于这些对象或变形，可能无法将其关联到骨架模型。因此，我们的方法具有三重新颖性：首先，我们提出并推导了用于非刚性姿态估计和分割任务的梯度流。其次，由于训练集的可能非线性结构，我们针对形状分析任务演化通过核 PCA 获得的原像。第三，我们表明形状权重的推导是通用的。这使我们可以使用各种核以及其他统计学习方法，而无需对整体形状演化方案进行大量更改。与其他技术相比，我们使用有限维优化方案来解决非刚性问题，这是一个无限维任务。更重要的是，我们不需要显式地知道各种形状之间的相互作用，例如骨架模型所需的相互作用，因为这是通过形状学习隐式完成的。我们在几个具有挑战性的姿态估计和分割场景中提供了实验结果。
