Department of Geomatics, National Cheng-Kung University, Tainan City 701, Taiwan.
Sensors (Basel). 2022 Aug 16;22(16):6111. doi: 10.3390/s22166111.
Stereo image dense matching, which plays a key role in 3D reconstruction, remains a challenging task in photogrammetry and computer vision. In addition to block-based matching, recent studies based on artificial neural networks have achieved great progress in stereo matching by using deep convolutional networks. This study proposes a novel network called a dual guided aggregation network (Dual-GANet), which utilizes both left-to-right and right-to-left image matchings in network design and training to reduce the possibility of pixel mismatch. Flipped training with a cost volume consistentization is introduced to realize the learning of invisible-to-visible pixel matching and left−right consistency matching. In addition, suppressed multi-regression is proposed, which suppresses unrelated information before regression and selects multiple peaks from a disparity probability distribution. The proposed dual network with the left−right consistent matching scheme can be applied to most stereo matching models. To estimate the performance, GANet, which is designed based on semi-global matching, was selected as the backbone with extensions and modifications on guided aggregation, disparity regression, and loss function. Experimental results on the SceneFlow and KITTI2015 datasets demonstrate the superiority of the Dual-GANet compared to related models in terms of average end-point-error (EPE) and pixel error rate (ER). The Dual-GANet with an average EPE performance = 0.418 and ER (>1 pixel) = 5.81% for SceneFlow and average EPE = 0.589 and ER (>3 pixels) = 1.76% for KITTI2005 is better than the backbone model with the average EPE performance of = 0.440 and ER (>1 pixel) = 6.56% for SceneFlow and average EPE = 0.790 and ER (>3 pixels) = 2.32% for KITTI2005.
立体图像密集匹配在 3D 重建中起着关键作用，仍然是摄影测量和计算机视觉中的一项具有挑战性的任务。除了基于块的匹配外，基于人工神经网络的最新研究通过使用深度卷积网络在立体匹配方面取得了很大的进展。本研究提出了一种名为双引导聚合网络（Dual-GANet）的新网络，该网络在网络设计和训练中利用左右图像匹配来减少像素不匹配的可能性。引入翻转训练和代价体一致性化，以实现不可见-可见像素匹配和左右一致性匹配的学习。此外，提出了抑制多回归，在回归之前抑制不相关的信息，并从视差概率分布中选择多个峰值。所提出的具有左右一致匹配方案的双网络可以应用于大多数立体匹配模型。为了评估性能，选择基于半全局匹配设计的 GANet 作为骨干，并在引导聚合、视差回归和损失函数方面进行扩展和修改。在 SceneFlow 和 KITTI2015 数据集上的实验结果表明，与相关模型相比，Dual-GANet 在平均端点误差（EPE）和像素误差率（ER）方面具有优越性。对于 SceneFlow，Dual-GANet 的平均 EPE 性能为 0.418，ER (>1 像素) 为 5.81%，ER (>3 像素) 为 1.76%；对于 KITTI2005，Dual-GANet 的平均 EPE 性能为 0.589，ER (>1 像素) 为 6.56%，ER (>3 像素) 为 1.76%。优于骨干模型，对于 SceneFlow，其平均 EPE 性能为 0.440，ER (>1 像素) 为 6.56%，ER (>3 像素) 为 2.32%；对于 KITTI2005，其平均 EPE 性能为 0.790，ER (>1 像素) 为 2.32%，ER (>3 像素) 为 2.32%。
