Department of Electronic Engineering, Tsinghua University, Beijing 100084, China.
Department of Mathematics, University of Iowa, Iowa City, IA 52242, USA.
Sensors (Basel). 2019 Apr 10;19(7):1714. doi: 10.3390/s19071714.
Deep learning models combining spectral and spatial features have been proven to be effective for hyperspectral image (HSI) classification. However, most spatial feature integration methods only consider a single input spatial scale regardless of various shapes and sizes of objects over the image plane, leading to missing scale-dependent information. In this paper, we propose a hierarchical multi-scale convolutional neural networks (CNNs) with auxiliary classifiers (HMCNN-AC) to learn hierarchical multi-scale spectral-spatial features for HSI classification. First, to better exploit the spatial information, multi-scale image patches for each pixel are generated at different spatial scales. These multi-scale patches are all centered at the same central spectrum but with shrunken spatial scales. Then, we apply multi-scale CNNs to extract spectral-spatial features from each scale patch. The obtained multi-scale convolutional features are considered as structured sequential data with spectral-spatial dependency, and a bidirectional LSTM is proposed to capture the correlation and extract a hierarchical representation for each pixel. To better train the whole network, weighted auxiliary classifiers are employed for the multi-scale CNNs and optimized together with the main loss function. Experimental results on three public HSI datasets demonstrate the superiority of our proposed framework over some state-of-the-art methods.
深度学习模型结合光谱和空间特征已被证明对高光谱图像（HSI）分类有效。然而，大多数空间特征集成方法仅考虑单一输入空间尺度，而不考虑图像平面上各种形状和大小的目标，导致缺少与尺度相关的信息。在本文中，我们提出了一种具有辅助分类器的分层多尺度卷积神经网络（HMCNN-AC），用于学习 HSI 分类的分层多尺度光谱-空间特征。首先，为了更好地利用空间信息，针对每个像素在不同的空间尺度上生成多尺度图像补丁。这些多尺度补丁都以相同的中心光谱为中心，但空间尺度缩小了。然后，我们应用多尺度 CNN 从每个尺度补丁中提取光谱-空间特征。获得的多尺度卷积特征被视为具有光谱-空间依赖性的结构化顺序数据，并提出了双向 LSTM 来捕获相关性并为每个像素提取分层表示。为了更好地训练整个网络，我们为多尺度 CNN 采用了加权辅助分类器，并与主要损失函数一起进行优化。在三个公共 HSI 数据集上的实验结果表明，我们提出的框架优于一些最先进的方法。
