School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou, Gansu, China.
School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu, China.
PLoS One. 2023 Sep 18;18(9):e0290231. doi: 10.1371/journal.pone.0290231. eCollection 2023.
Infrared and visible image fusion can generate a fusion image with clear texture and prominent goals under extreme conditions. This capability is important for all-day climate detection and other tasks. However, most existing fusion methods for extracting features from infrared and visible images are based on convolutional neural networks (CNNs). These methods often fail to make full use of the salient objects and texture features in the raw image, leading to problems such as insufficient texture details and low contrast in the fused images. To this end, we propose an unsupervised end-to-end Fusion Decomposition Network (FDNet) for infrared and visible image fusion. Firstly, we construct a fusion network that extracts gradient and intensity information from raw images, using multi-scale layers, depthwise separable convolution, and improved convolution block attention module (I-CBAM). Secondly, as the FDNet network is based on the gradient and intensity information of the image for feature extraction, gradient and intensity loss are designed accordingly. Intensity loss adopts the improved Frobenius norm to adjust the weighing values between the fused image and the two raw to select more effective information. The gradient loss introduces an adaptive weight block that determines the optimized objective based on the richness of texture information at the pixel scale, ultimately guiding the fused image to generate more abundant texture information. Finally, we design a single and dual channel convolutional layer decomposition network, which keeps the decomposed image as possible with the input raw image, forcing the fused image to contain richer detail information. Compared with various other representative image fusion methods, our proposed method not only has good subjective vision, but also achieves advanced fusion performance in objective evaluation.
红外与可见光图像融合可以在极端条件下生成具有清晰纹理和突出目标的融合图像。这种能力对于全天候气候检测和其他任务非常重要。然而，大多数从红外和可见光图像中提取特征的现有融合方法都是基于卷积神经网络（CNN）的。这些方法往往无法充分利用原始图像中的显著目标和纹理特征，导致融合图像的纹理细节不足和对比度低等问题。为此，我们提出了一种用于红外与可见光图像融合的无监督端到端融合分解网络（FDNet）。首先，我们构建了一个融合网络，该网络使用多尺度层、深度可分离卷积和改进的卷积块注意力模块（I-CBAM）从原始图像中提取梯度和强度信息。其次，由于 FDNet 网络是基于图像的梯度和强度信息进行特征提取的，因此相应地设计了梯度和强度损失。强度损失采用改进的 Frobenius 范数来调整融合图像和两个原始图像之间的加权值，以选择更有效的信息。梯度损失引入了一个自适应权重块，根据像素级纹理信息的丰富程度确定优化目标，最终引导融合图像生成更丰富的纹理信息。最后，我们设计了一个单通道和双通道卷积层分解网络，该网络尽可能地保持分解后的图像与输入的原始图像相似，迫使融合后的图像包含更丰富的细节信息。与各种其他有代表性的图像融合方法相比，我们提出的方法不仅具有良好的主观视觉效果，而且在客观评价中也实现了先进的融合性能。
