Hou Jilei, Zhang Dazhi, Wu Wei, Ma Jiayi, Zhou Huabing
College of Computer Science and Engineering, Wuhan Institute of Technology, Wuhan 430205, China.
Research Institute of Nuclear Power Operation, Wuhan 430000, China.
Entropy (Basel). 2021 Mar 21;23(3):376. doi: 10.3390/e23030376.
This paper proposes a new generative adversarial network for infrared and visible image fusion based on semantic segmentation (SSGAN), which can consider not only the low-level features of infrared and visible images, but also the high-level semantic information. Source images can be divided into foregrounds and backgrounds by semantic masks. The generator with a dual-encoder-single-decoder framework is used to extract the feature of foregrounds and backgrounds by different encoder paths. Moreover, the discriminator's input image is designed based on semantic segmentation, which is obtained by combining the foregrounds of the infrared images with the backgrounds of the visible images. Consequently, the prominence of thermal targets in the infrared images and texture details in the visible images can be preserved in the fused images simultaneously. Qualitative and quantitative experiments on publicly available datasets demonstrate that the proposed approach can significantly outperform the state-of-the-art methods.
本文提出了一种基于语义分割的用于红外与可见光图像融合的新型生成对抗网络（SSGAN），该网络不仅可以考虑红外和可见光图像的低级特征，还能考虑高级语义信息。源图像可以通过语义掩码分为前景和背景。具有双编码器-单解码器框架的生成器用于通过不同的编码器路径提取前景和背景的特征。此外，鉴别器的输入图像基于语义分割进行设计，它是通过将红外图像的前景与可见光图像的背景相结合而获得的。因此，红外图像中热目标的突出性和可见光图像中的纹理细节可以同时保留在融合图像中。在公开可用数据集上进行的定性和定量实验表明，所提出的方法能够显著优于现有最先进的方法。
