School of Electronic Engineering, Xi'an Shiyou University, Xi'an 710065, China.
State Key Laboratory of Advanced Design and Manufacturing for Vehicle Body, Hunan University, Changsha 410082, China.
Sensors (Basel). 2022 Jul 20;22(14):5430. doi: 10.3390/s22145430.
This paper presents an algorithm for infrared and visible image fusion using significance detection and Convolutional Neural Networks with the aim of integrating discriminatory features and improving the overall quality of visual perception. Firstly, a global contrast-based significance detection algorithm is applied to the infrared image, so that salient features can be extracted, highlighting high brightness values and suppressing low brightness values and image noise. Secondly, a special loss function is designed for infrared images to guide the extraction and reconstruction of features in the network, based on the principle of salience detection, while the more mainstream gradient loss is used as the loss function for visible images in the network. Afterwards, a modified residual network is applied to complete the extraction of features and image reconstruction. Extensive qualitative and quantitative experiments have shown that fused images are sharper and contain more information about the scene, and the fused results look more like high-quality visible images. The generalization experiments also demonstrate that the proposed model has the ability to generalize well, independent of the limitations of the sensor. Overall, the algorithm proposed in this paper performs better compared to other state-of-the-art methods.
本文提出了一种基于显著性检测和卷积神经网络的红外与可见光图像融合算法，旨在融合判别特征，提高视觉感知的整体质量。首先，对红外图像应用基于全局对比度的显著性检测算法，以提取显著特征，突出高亮度值，抑制低亮度值和图像噪声。其次，根据显著性检测原理，为红外图像设计了一种特殊的损失函数，以引导网络中特征的提取和重建，而网络中可见光图像的损失函数则采用更为主流的梯度损失函数。之后，应用改进的残差网络完成特征提取和图像重建。大量定性和定量实验表明，融合图像更加清晰，包含更多场景信息，融合结果更像是高质量的可见光图像。泛化实验也表明，与其他最先进的方法相比，所提出的模型具有很好的泛化能力，不受传感器限制的影响。总的来说，本文提出的算法性能优于其他现有的方法。
