College of Geodesy and Geomatics, Shandong University of Science and Technology, Qingdao 266590, China.
National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, Xi'an 710072, China.
Sensors (Basel). 2021 Apr 1;21(7):2416. doi: 10.3390/s21072416.
Feature matching plays a crucial role in the process of 3D reconstruction based on the structure from motion (SfM) technique. For a large collection of oblique images, feature matching is one of the most time-consuming steps, and the matching result directly affects the accuracy of subsequent tasks. Therefore, how to extract the reasonable feature points robustly and efficiently to improve the matching speed and quality has received extensive attention from scholars worldwide. Most studies perform quantitative feature point selection based on image Difference-of-Gaussian (DoG) pyramids in practice. However, the stability and spatial distribution of feature points are not considered enough, resulting in selected feature points that may not adequately reflect the scene structures and cannot guarantee the matching rate and the aerial triangulation accuracy. To address these issues, an improved method for stable feature point selection in SfM considering image semantic and structural characteristics is proposed. First, the visible-band difference vegetation index is used to identify the vegetation areas from oblique images, and the line feature in the image is extracted by the optimized line segment detector algorithm. Second, the feature point two-tuple classification model is established, in which the vegetation area recognition result is used as the semantic constraint, the line feature extraction result is used as the structural constraint, and the feature points are divided into three types. Finally, a progressive selection algorithm for feature points is proposed, in which feature points in the DoG pyramid are selected by classes and levels until the number of feature points is satisfied. Oblique images of a 40-km area in Dongying city, China, were used for validation. The experimental results show that compared to the state-of-the-art method, the method proposed in this paper not only effectively reduces the number of feature points but also better reflects the scene structure. At the same time, the average reprojection error of the aerial triangulation decrease by 20%, the feature point matching rate increase by 3%, the selected feature points are more stable and reasonable.
特征匹配在基于运动结构（SfM）技术的 3D 重建过程中起着至关重要的作用。对于大量倾斜图像，特征匹配是最耗时的步骤之一，匹配结果直接影响后续任务的准确性。因此，如何提取合理的特征点，提高匹配速度和质量，受到了全球学者的广泛关注。大多数研究在实践中基于图像差分高斯（DoG）金字塔进行定量特征点选择。然而，特征点的稳定性和空间分布没有得到足够的考虑，导致选择的特征点可能不能充分反映场景结构，无法保证匹配率和航空三角测量精度。针对这些问题，提出了一种改进的考虑图像语义和结构特征的 SfM 中稳定特征点选择方法。首先，利用可见波段差值植被指数从倾斜图像中识别植被区域，并采用优化的线段检测器算法提取图像中的线特征。其次，建立特征点二项式分类模型，其中植被区域识别结果作为语义约束，线特征提取结果作为结构约束，将特征点分为三类。最后，提出了一种基于渐进选择的特征点选择算法，按类和级别逐步选择 DoG 金字塔中的特征点，直到满足特征点数量。使用中国东营市 40 公里区域的倾斜图像进行验证。实验结果表明，与现有方法相比，本文提出的方法不仅有效地减少了特征点的数量，而且更好地反映了场景结构。同时，航空三角测量的平均重投影误差降低了 20%，特征点匹配率提高了 3%，选择的特征点更加稳定和合理。
