IEEE Trans Pattern Anal Mach Intell. 2022 Nov;44(11):7417-7435. doi: 10.1109/TPAMI.2021.3118833. Epub 2022 Oct 4.
We propose a three-stage 6 DoF object detection method called DPODv2 (Dense Pose Object Detector) that relies on dense correspondences. We combine a 2D object detector with a dense correspondence estimation network and a multi-view pose refinement method to estimate a full 6 DoF pose. Unlike other deep learning methods that are typically restricted to monocular RGB images, we propose a unified deep learning network allowing different imaging modalities to be used (RGB or Depth). Moreover, we propose a novel pose refinement method, that is based on differentiable rendering. The main concept is to compare predicted and rendered correspondences in multiple views to obtain a pose which is consistent with predicted correspondences in all views. Our proposed method is evaluated rigorously on different data modalities and types of training data in a controlled setup. The main conclusions is that RGB excels in correspondence estimation, while depth contributes to the pose accuracy if good 3D-3D correspondences are available. Naturally, their combination achieves the overall best performance. We perform an extensive evaluation and an ablation study to analyze and validate the results on several challenging datasets. DPODv2 achieves excellent results on all of them while still remaining fast and scalable independent of the used data modality and the type of training data.
我们提出了一种称为 DPODv2（密集姿态对象检测）的三阶段 6DoF 对象检测方法，该方法依赖于密集对应关系。我们将 2D 对象检测器与密集对应估计网络和多视图姿态细化方法相结合，以估计完整的 6DoF 姿态。与其他通常仅限于单目 RGB 图像的深度学习方法不同，我们提出了一种统一的深度学习网络，允许使用不同的成像模式（RGB 或深度）。此外，我们提出了一种新颖的姿态细化方法，该方法基于可微分渲染。主要思想是在多个视图中比较预测的和渲染的对应关系，以获得与所有视图中预测的对应关系一致的姿态。我们在受控设置下，使用不同的数据模态和类型的训练数据对提出的方法进行了严格的评估。主要结论是 RGB 在对应估计方面表现出色，而深度如果有良好的 3D-3D 对应关系，则有助于提高姿态准确性。自然地，它们的组合实现了整体最佳性能。我们进行了广泛的评估和消融研究，以在几个具有挑战性的数据集上分析和验证结果。DPODv2 在所有这些数据集上都取得了优异的结果，同时仍然保持快速和可扩展，与所使用的数据模态和训练数据的类型无关。
