一种基于通道注意力网络的多光谱滤波器阵列图像快照多光谱去马赛克方法。
A Snapshot Multi-Spectral Demosaicing Method for Multi-Spectral Filter Array Images Based on Channel Attention Network.
作者信息
Zhang Xuejun, Dai Yidan, Zhang Geng, Zhang Xuemin, Hu Bingliang
机构信息
Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, China.
University of Chinese Academy of Sciences, Beijing 100049, China.
出版信息
Sensors (Basel). 2024 Feb 1;24(3):943. doi: 10.3390/s24030943.
Multi-spectral imaging technologies have made great progress in the past few decades. The development of snapshot cameras equipped with a specific multi-spectral filter array (MSFA) allow dynamic scenes to be captured on a miniaturized platform across multiple spectral bands, opening up extensive applications in quantitative and visualized analysis. However, a snapshot camera based on MSFA captures a single band per pixel; thus, the other spectral band components of pixels are all missed. The raw images, which are captured by snapshot multi-spectral imaging systems, require a reconstruction procedure called demosaicing to estimate a fully defined multi-spectral image (MSI). With increasing spectral bands, the challenge of demosaicing becomes more difficult. Furthermore, the existing demosaicing methods will produce adverse artifacts and aliasing because of the adverse effects of spatial interpolation and the inadequacy of the number of layers in the network structure. In this paper, a novel multi-spectral demosaicing method based on a deep convolution neural network (CNN) is proposed for the reconstruction of full-resolution multi-spectral images from raw MSFA-based spectral mosaic images. The CNN is integrated with the channel attention mechanism to protect important channel features. We verify the merits of the proposed method using 5 × 5 raw mosaic images on synthetic as well as real-world data. The experimental results show that the proposed method outperforms the existing demosaicing methods in terms of spatial details and spectral fidelity.
在过去几十年中,多光谱成像技术取得了巨大进展。配备特定多光谱滤波器阵列(MSFA)的快照相机的发展,使得动态场景能够在小型化平台上跨多个光谱波段进行捕捉,为定量和可视化分析开辟了广泛的应用。然而,基于MSFA的快照相机每个像素仅捕获一个波段;因此,像素的其他光谱波段成分全部丢失。由快照多光谱成像系统捕获的原始图像需要一种称为去马赛克的重建过程来估计一个完整定义的多光谱图像(MSI)。随着光谱波段数量的增加,去马赛克的挑战变得更加困难。此外,由于空间插值的不利影响和网络结构中层数的不足,现有的去马赛克方法会产生不良伪像和混叠。本文提出了一种基于深度卷积神经网络(CNN)的新型多光谱去马赛克方法,用于从基于MSFA的原始光谱镶嵌图像重建全分辨率多光谱图像。CNN与通道注意力机制相结合以保护重要的通道特征。我们使用5×5原始镶嵌图像在合成数据和真实世界数据上验证了所提出方法的优点。实验结果表明,所提出的方法在空间细节和光谱保真度方面优于现有的去马赛克方法。