Zhao Yuzhi, Po Lai-Man, Lin Tingyu, Yan Qiong, Liu Wei, Xian Pengfei
IEEE Trans Neural Netw Learn Syst. 2024 Dec;35(12):17137-17150. doi: 10.1109/TNNLS.2023.3300099. Epub 2024 Dec 2.
Hyperspectral (HS) reconstruction from RGB images denotes the recovery of whole-scene HS information, which has attracted much attention recently. State-of-the-art approaches often adopt convolutional neural networks to learn the mapping for HS reconstruction from RGB images. However, they often do not achieve high HS reconstruction performance across different scenes consistently. In addition, their performance in recovering HS images from clean and real-world noisy RGB images is not consistent. To improve the HS reconstruction accuracy and robustness across different scenes and from different input images, we present an effective HSGAN framework with a two-stage adversarial training strategy. The generator is a four-level top-down architecture that extracts and combines features on multiple scales. To generalize well to real-world noisy images, we further propose a spatial-spectral attention block (SSAB) to learn both spatial-wise and channel-wise relations. We conduct the HS reconstruction experiments from both clean and real-world noisy RGB images on five well-known HS datasets. The results demonstrate that HSGAN achieves superior performance to existing methods. Please visit https://github.com/zhaoyuzhi/HSGAN to try our codes.
从RGB图像进行高光谱（HS）重建旨在恢复整个场景的HS信息，这一领域近来备受关注。当前的先进方法通常采用卷积神经网络来学习从RGB图像进行HS重建的映射。然而，它们往往无法在不同场景下始终如一地实现较高的HS重建性能。此外，它们从干净的和真实世界的噪声RGB图像中恢复HS图像的性能也不一致。为了提高跨不同场景以及从不同输入图像进行HS重建的准确性和鲁棒性，我们提出了一种有效的具有两阶段对抗训练策略的HSGAN框架。生成器是一个四级自上而下的架构，可在多个尺度上提取并组合特征。为了能很好地推广到真实世界的噪声图像，我们进一步提出了一种空间光谱注意力模块（SSAB）来学习空间维度和通道维度的关系。我们在五个著名的HS数据集上对干净的和真实世界的噪声RGB图像都进行了HS重建实验。结果表明，HSGAN的性能优于现有方法。请访问https://github.com/zhaoyuzhi/HSGAN来试用我们的代码。
