Luo Yu, Cha Hao, Zuo Lei, Cheng Peng, Zhao Qing
College of Electronic Engineering, Naval University of Engineering, Wuhan, 4300000, China.
Sci Rep. 2023 Aug 9;13(1):12941. doi: 10.1038/s41598-023-39863-3.
In all-day-all-weather tasks, well-aligned multi-modality images pairs can provide extensive complementary information for image-guided UAV target detection. However, multi-modality images in real scenarios are often misaligned, and images registration is extremely difficult due to spatial deformation and the difficulty narrowing cross-modality discrepancy. To better overcome the obstacle, in this paper, we construct a General Cross-Modality Registration (GCMR) Framework, which explores generation registration pattern to simplify the cross-modality image registration into a easier mono-modality image registration with an Image Cross-Modality Translation Network (ICMTN) module and a Multi-level Residual Dense Registration Network (MRDRN). Specifically, ICMTN module is used to generate a pseudo infrared image taking a visible image as input and correct the distortion of structural information during the translation of image modalities. Benefiting from the favorable geometry correct ability of the ICMTN, we further employs MRDRN module which can fully extract and exploit the mutual information of misaligned images to better registered Visible and Infrared image in a mono-modality setting. We evaluate five variants of our approach on the public Anti-UAV datasets. The extensive experimental results demonstrate that the proposed architecture achieves state-of-the-art performance.
在全天候任务中，对齐良好的多模态图像对可为图像引导的无人机目标检测提供广泛的补充信息。然而，实际场景中的多模态图像往往未对齐，并且由于空间变形和缩小跨模态差异的困难，图像配准极具挑战性。为了更好地克服这一障碍，在本文中，我们构建了一个通用跨模态配准（GCMR）框架，该框架探索生成配准模式，通过图像跨模态转换网络（ICMTN）模块和多级残差密集配准网络（MRDRN）将跨模态图像配准简化为更简单的单模态图像配准。具体而言，ICMTN模块用于以可见光图像为输入生成伪红外图像，并在图像模态转换过程中校正结构信息的失真。受益于ICMTN良好的几何校正能力，我们进一步采用MRDRN模块，该模块可以充分提取和利用未对齐图像的互信息，以便在单模态设置中更好地配准可见光和红外图像。我们在公开的反无人机数据集上评估了我们方法的五个变体。大量实验结果表明，所提出的架构实现了最优性能。
