IEEE Trans Pattern Anal Mach Intell. 2023 Mar;45(3):2849-2863. doi: 10.1109/TPAMI.2022.3174130. Epub 2023 Feb 3.
Homography estimation is a basic image alignment method in many applications. It is usually done by extracting and matching sparse feature points, which are error-prone in low-light and low-texture images. On the other hand, previous deep homography approaches use either synthetic images for supervised learning or aerial images for unsupervised learning, both ignoring the importance of handling depth disparities and moving objects in real-world applications. To overcome these problems, in this work, we propose an unsupervised deep homography method with a new architecture design. In the spirit of the RANSAC procedure in traditional methods, we specifically learn an outlier mask to only select reliable regions for homography estimation. We calculate loss with respect to our learned deep features instead of directly comparing image content as did previously. To achieve the unsupervised training, we also formulate a novel triplet loss customized for our network. We verify our method by conducting comprehensive comparisons on a new dataset that covers a wide range of scenes with varying degrees of difficulties for the task. Experimental results reveal that our method outperforms the state-of-the-art, including deep solutions and feature-based solutions.
单应估计是许多应用中基本的图像对齐方法。它通常通过提取和匹配稀疏特征点来完成，但在低光照和低纹理图像中，这种方法容易出错。另一方面，以前的深度单应估计方法要么使用合成图像进行监督学习，要么使用航空图像进行无监督学习，这两种方法都忽略了处理现实应用中深度差异和移动物体的重要性。为了解决这些问题，在这项工作中，我们提出了一种具有新架构设计的无监督深度单应估计方法。本着传统方法中 RANSAC 过程的精神，我们专门学习了一个异常值掩模，只选择可靠的区域进行单应估计。我们计算相对于我们学习的深度特征的损失，而不是像以前那样直接比较图像内容。为了实现无监督训练，我们还为我们的网络制定了一种新的定制三元组损失。我们在一个新的数据集上进行了全面的比较，该数据集涵盖了广泛的场景，任务难度也各不相同。实验结果表明，我们的方法优于最先进的方法，包括深度解决方案和基于特征的解决方案。
