Suppr超能文献

基于事件相机的遥感与广域运动图像姿态优化(EC-WAMI)

EC-WAMI: Event Camera-Based Pose Optimization in Remote Sensing and Wide-Area Motion Imagery.

作者信息

Nkrumah Isaac, Moshrefizadeh Maryam, Tahri Omar, Blasch Erik, Palaniappan Kannappan, AliAkbarpour Hadi

机构信息

Artificial Intelligence and Robotics Lab (AIRLab), Department of Computer Science, Saint Louis University, Saint Louis, MO 63103, USA.

CNRS UMR 6303 ICB, Université de Bourgogne, 21078 Dijon, France.

出版信息

Sensors (Basel). 2024 Nov 24;24(23):7493. doi: 10.3390/s24237493.

Abstract

In this paper, we present , the first successful application of neuromorphic for Wide-Area Motion Imagery (WAMI) and Remote Sensing (RS), showcasing their potential for advancing Structure-from-Motion (SfM) and 3D reconstruction across diverse imaging scenarios. ECs, which detect asynchronous pixel-level , offer key advantages over traditional frame-based sensors such as high temporal resolution, low power consumption, and resilience to dynamic lighting. These capabilities allow ECs to overcome challenges such as glare, uneven lighting, and low-light conditions that are common in aerial imaging and remote sensing, while also extending UAV flight endurance. To evaluate the effectiveness of ECs in WAMI, we simulate event data from RGB WAMI imagery and integrate them into SfM pipelines for camera pose optimization and 3D point cloud generation. Using two state-of-the-art SfM methods, namely, COLMAP and Bundle Adjustment for Sequential Imagery (BA4S), we show that although ECs do not capture scene content like traditional cameras, their spike-based events, which only measure , allow for accurate camera pose recovery in WAMI scenarios even in low-framerate(5 fps) simulations. Our results indicate that while BA4S and COLMAP provide comparable accuracy, BA4S significantly outperforms COLMAP in terms of speed. Moreover, we evaluate different feature extraction methods, showing that the deep learning-based LIGHTGLUE descriptor consistently outperforms traditional handcrafted descriptors by providing improved reliability and accuracy of event-based SfM. These results highlight the broader potential of ECs in remote sensing, aerial imaging, and 3D reconstruction beyond conventional WAMI applications. Our dataset will be made available for public use.

摘要

在本文中,我们展示了神经形态技术在广域运动图像(WAMI)和遥感(RS)中的首次成功应用,展示了它们在推进跨各种成像场景的运动结构(SfM)和3D重建方面的潜力。事件相机(EC)能够检测异步像素级事件,与传统的基于帧的传感器相比具有关键优势,如高时间分辨率、低功耗以及对动态光照的适应性。这些能力使事件相机能够克服诸如眩光、光照不均和低光照条件等在航空成像和遥感中常见的挑战,同时还能延长无人机的飞行续航时间。为了评估事件相机在广域运动图像中的有效性,我们模拟了来自RGB广域运动图像的事件数据,并将其集成到SfM管道中以进行相机姿态优化和3D点云生成。使用两种先进的SfM方法,即COLMAP和顺序图像束调整(BA4S),我们表明,尽管事件相机不像传统相机那样捕捉场景内容,但其基于脉冲的事件(仅测量[此处原文缺失相关内容])即使在低帧率(5帧/秒)模拟的广域运动图像场景中也能实现准确的相机姿态恢复。我们的结果表明,虽然BA4S和COLMAP提供了相当的精度,但BA4S在速度方面明显优于COLMAP。此外,我们评估了不同的特征提取方法,表明基于深度学习的LIGHTGLUE描述符通过提高基于事件的SfM的可靠性和准确性,始终优于传统的手工描述符。这些结果突出了事件相机在遥感、航空成像和3D重建中超越传统广域运动图像应用的更广泛潜力。我们的数据集将可供公众使用。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/a7cd/11644785/bb289fd56a50/sensors-24-07493-g001.jpg

文献检索

告别复杂PubMed语法,用中文像聊天一样搜索,搜遍4000万医学文献。AI智能推荐,让科研检索更轻松。

立即免费搜索

文件翻译

保留排版,准确专业,支持PDF/Word/PPT等文件格式,支持 12+语言互译。

免费翻译文档

深度研究

AI帮你快速写综述,25分钟生成高质量综述,智能提取关键信息,辅助科研写作。

立即免费体验