Picon Artzai, Galan Pablo, Bereciartua-Perez Arantza, Benito-Del-Valle Leire
TECNALIA, Basque Research and Technology Alliance (BRTA), Parque Tecnológico de Bizkaia, C/ Geldo. Edificio 700, E-48160, Derio - Bizkaia, Spain; University of the Basque Country, Plaza Torres Quevedo, 48013 Bilbao, Spain.
TECNALIA, Basque Research and Technology Alliance (BRTA), Parque Tecnológico de Bizkaia, C/ Geldo. Edificio 700, E-48160, Derio - Bizkaia, Spain.
Spectrochim Acta A Mol Biomol Spectrosc. 2025 Apr 5;330:125665. doi: 10.1016/j.saa.2024.125665. Epub 2024 Dec 30.
Hyperspectral imaging, a rapidly evolving field, has witnessed the ascendancy of deep learning techniques, supplanting classical feature extraction and classification methods in various applications. However, many researchers employ arbitrary architectures for hyperspectral image processing, often without rigorous analysis of the interplay between spectral and spatial information. This oversight neglects the implications of combining these two modalities on model performance, consumption, and inference time. This paper evaluates the impact of including different spatial (visual texture) and spectral (captured spectral information) features on deep learning architectures for hyperspectral image segmentation. To this end, it presents different architectural configurations with varying levels of spectral and spatial information and are evaluated in terms of identification performance, energy consumption, and inference time. Additionally, the transferability of knowledge from large pre-trained image foundation models, originally designed for RGB images, to the hyperspectral domain is explored. Results show that incorporating spatial information alongside spectral data leads to improved segmentation results. However, not all spectral wavelengths are necessary to obtain the optimal performance/energy consumption ratio, which is required for faster and more carbon-neutral models. Training foundation models from the RGB domain leads to lower performance and higher energy consumption models with longer inference times. It is also essential to further develop novel architectures that integrate spectral and spatial information and adapt RGB foundation models to the hyperspectral domain. Furthermore, this paper contributes to the field by cleaning and publicly releasing the Tecnalia WEEE Hyperspectral dataset. This dataset contains different non-ferrous fractions of Waste Electrical and Electronic Equipment (WEEE), including Copper, Brass, Aluminum, Stainless Steel, and White Copper, spanning the range of 400 to 1000 nm.
高光谱成像作为一个快速发展的领域，见证了深度学习技术的崛起，在各种应用中取代了传统的特征提取和分类方法。然而，许多研究人员在处理高光谱图像时采用任意架构，往往没有对光谱信息和空间信息之间的相互作用进行严格分析。这种疏忽忽略了将这两种模态结合起来对模型性能、消耗和推理时间的影响。本文评估了包含不同空间（视觉纹理）和光谱（捕获的光谱信息）特征对用于高光谱图像分割的深度学习架构的影响。为此，本文提出了具有不同光谱和空间信息水平的不同架构配置，并在识别性能、能耗和推理时间方面进行了评估。此外，还探索了从最初为RGB图像设计的大型预训练图像基础模型向高光谱领域的知识可转移性。结果表明，将空间信息与光谱数据一起纳入可提高分割结果。然而，并非所有光谱波长对于获得更快、更碳中和模型所需的最佳性能/能耗比都是必要的。从RGB领域训练基础模型会导致性能较低、能耗较高且推理时间较长的模型。进一步开发整合光谱和空间信息的新型架构，并使RGB基础模型适应高光谱领域也至关重要。此外，本文通过清理并公开发布Tecnalia WEEE高光谱数据集为该领域做出了贡献。该数据集包含废弃电子电气设备（WEEE）的不同有色金属部分，包括铜、黄铜、铝、不锈钢和白铜，光谱范围为400至1000纳米。
