Tragakis Athanasios, Kaul Chaitanya, Mitchell Kevin J, Dai Hang, Murray-Smith Roderick, Faccio Daniele
School of Physics and Astronomy, University of Glasgow, Glasgow G12 8QQ, UK.
School of Computing Science, University of Glasgow, Glasgow G12 8QQ, UK.
Sensors (Basel). 2024 Dec 24;25(1):24. doi: 10.3390/s25010024.
Accurate depth estimation is crucial for many fields, including robotics, navigation, and medical imaging. However, conventional depth sensors often produce low-resolution (LR) depth maps, making detailed scene perception challenging. To address this, enhancing LR depth maps to high-resolution (HR) ones has become essential, guided by HR-structured inputs like RGB or grayscale images. We propose a novel sensor fusion methodology for guided depth super-resolution (GDSR), a technique that combines LR depth maps with HR images to estimate detailed HR depth maps. Our key contribution is the Incremental guided attention fusion (IGAF) module, which effectively learns to fuse features from RGB images and LR depth maps, producing accurate HR depth maps. Using IGAF, we build a robust super-resolution model and evaluate it on multiple benchmark datasets. Our model achieves state-of-the-art results compared to all baseline models on the NYU v2 dataset for ×4, ×8, and ×16 upsampling. It also outperforms all baselines in a zero-shot setting on the Middlebury, Lu, and RGB-D-D datasets. Code, environments, and models are available on GitHub.
精确的深度估计对于包括机器人技术、导航和医学成像在内的许多领域至关重要。然而，传统的深度传感器通常会生成低分辨率（LR）深度图，这使得详细的场景感知具有挑战性。为了解决这个问题，在RGB或灰度图像等高分辨率（HR）结构输入的引导下，将低分辨率深度图增强为高分辨率深度图变得至关重要。我们提出了一种用于引导深度超分辨率（GDSR）的新型传感器融合方法，该技术将低分辨率深度图与高分辨率图像相结合，以估计详细的高分辨率深度图。我们的关键贡献是增量引导注意力融合（IGAF）模块，它有效地学习融合来自RGB图像和低分辨率深度图的特征，从而生成准确的高分辨率深度图。使用IGAF，我们构建了一个强大的超分辨率模型，并在多个基准数据集上对其进行评估。在纽约大学v2数据集上进行×4、×8和×16上采样时，我们的模型与所有基线模型相比取得了领先的结果。在米德尔伯里、卢和RGB-D-D数据集的零样本设置中，它也优于所有基线。代码、环境和模型可在GitHub上获取。
