Wang Jiashuo, Chen Yong, Sun Xiaoyun, Xing Hui, Zhang Fan, Song Shiji, Yu Shuyong
School of Mechanical Engineering, Shijiazhuang Tiedao University, Shijiazhuang 050043, Hebei, China.
Beijing Railway Signal Co., Ltd., Daxing, Beijing 102613, China.
iScience. 2024 Sep 10;27(10):110915. doi: 10.1016/j.isci.2024.110915. eCollection 2024 Oct 18.
Infrared and visible image fusion aims to produce images that highlight key targets and offer distinct textures, by merging the thermal radiation infrared images with the detailed texture visible images. Traditional auto encoder-decoder-based fusion methods often rely on manually designed fusion strategies, which lack flexibility across different scenarios. Addressing this limitation, we introduce EMAFusion, a fusion approach featuring an enhanced multiscale encoder and a learnable, lightweight fusion network. Our method incorporates skip connections, the convolutional block attention module (CBAM), and nest architecture within the auto encoder-decoder framework to adeptly extract and preserve multiscale features for fusion tasks. Furthermore, a fusion network driven by spatial and channel attention mechanisms is proposed, designed to precisely capture and integrate essential features from both image types. Comprehensive evaluations of the TNO image fusion dataset affirm the proposed method's superiority over existing state-of-the-art techniques, demonstrating its potential for advancing infrared and visible image fusion.
红外与可见光图像融合旨在通过将热辐射红外图像与具有详细纹理的可见光图像进行合并，生成突出关键目标并提供清晰纹理的图像。传统的基于自动编码器-解码器的融合方法通常依赖于手动设计的融合策略，在不同场景下缺乏灵活性。为了解决这一局限性，我们引入了EMAFusion，这是一种融合方法，其特点是具有增强的多尺度编码器和可学习的轻量级融合网络。我们的方法在自动编码器-解码器框架内结合了跳跃连接、卷积块注意力模块（CBAM）和嵌套架构，以巧妙地提取和保留用于融合任务的多尺度特征。此外，还提出了一种由空间和通道注意力机制驱动的融合网络，旨在精确捕捉和整合来自两种图像类型的关键特征。对TNO图像融合数据集的综合评估证实了所提出方法优于现有的最先进技术，展示了其在推进红外与可见光图像融合方面的潜力。
