School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK.
School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China.
Sensors (Basel). 2023 Apr 18;23(8):4063. doi: 10.3390/s23084063.
Accurate and robust camera pose estimation is essential for high-level applications such as augmented reality and autonomous driving. Despite the development of global feature-based camera pose regression methods and local feature-based matching guided pose estimation methods, challenging conditions, such as illumination changes and viewpoint changes, as well as inaccurate keypoint localization, continue to affect the performance of camera pose estimation. In this paper, we propose a novel relative camera pose regression framework that uses global features with rotation consistency and local features with rotation invariance. First, we apply a multi-level deformable network to detect and describe local features, which can learn appearances and gradient information sensitive to rotation variants. Second, we process the detection and description processes using the results from pixel correspondences of the input image pairs. Finally, we propose a novel loss that combines relative regression loss and absolute regression loss, incorporating global features with geometric constraints to optimize the pose estimation model. Our extensive experiments report satisfactory accuracy on the 7Scenes dataset with an average mean translation error of 0.18 m and a rotation error of 7.44° using image pairs as input. Ablation studies were also conducted to verify the effectiveness of the proposed method in the tasks of pose estimation and image matching using the 7Scenes and HPatches datasets.
准确而稳健的相机位姿估计对于高级应用（如增强现实和自动驾驶）至关重要。尽管已经开发了基于全局特征的相机位姿回归方法和基于局部特征的匹配引导位姿估计方法，但挑战性的条件，如光照变化和视角变化，以及不准确的关键点定位，仍然会影响相机位姿估计的性能。在本文中，我们提出了一种新的相对相机位姿回归框架，该框架使用具有旋转一致性的全局特征和具有旋转不变性的局部特征。首先，我们应用多层可变形网络来检测和描述局部特征，该特征可以学习对旋转变体敏感的外观和梯度信息。其次，我们使用输入图像对的像素对应关系的结果来处理检测和描述过程。最后，我们提出了一种新的损失函数，它结合了相对回归损失和绝对回归损失，将具有几何约束的全局特征纳入其中，以优化位姿估计模型。我们在 7Scenes 数据集上进行了广泛的实验，报告了令人满意的精度，平均平移误差为 0.18 米，旋转误差为 7.44°，输入的是图像对。我们还进行了消融研究，以验证该方法在使用 7Scenes 和 HPatches 数据集进行位姿估计和图像匹配任务中的有效性。
