Gao Qian, Shen Xukun
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing 100191, China.
School of New Media Art and Design, Beihang University, Beijing 100191, China.
Sensors (Basel). 2021 Mar 24;21(7):2272. doi: 10.3390/s21072272.
Recovering height information from a single aerial image is a key problem in the fields of computer vision and remote sensing. At present, supervised learning methods have achieved impressive results, but, due to domain bias, the trained model cannot be directly applied to a new scene. In this paper, we propose a novel semi-supervised framework, StyHighNet, for accurately estimating the height of a single aerial image in a new city that requires only a small number of labeled data. The core is to transfer multi-source images to a unified style, making the unlabeled data provide the appearance distribution as additional supervision signals. The framework mainly contains three sub-networks: (1) the style transferring sub-network maps multi-source images into unified style distribution maps (USDMs); (2) the height regression sub-network, with the function of predicting the height maps from USDMs; and (3) the style discrimination sub-network, used to distinguish the sources of USDMs. Among them, the style transferring sub-network shoulders dual responsibilities: On the one hand, it needs to compute USDMs with obvious characteristics, so that the height regression sub-network can accurately estimate the height maps. On the other hand, it is necessary that the USDMs have consistent distribution to confuse the style discrimination sub-network, so as to achieve the goal of domain adaptation. Unlike previous methods, our style distribution function is learned unsupervised, thus it is of greater flexibility and better accuracy. Furthermore, when the style discrimination sub-network is shielded, this framework can also be used for supervised learning. We performed qualitatively and quantitative evaluations on two sets of public data, Vaihingen and Potsdam. Experiments show that the framework achieved superior performance in both supervised and semi-supervised learning modes.
从单张航空图像中恢复高度信息是计算机视觉和遥感领域的一个关键问题。目前，监督学习方法已经取得了令人瞩目的成果，但由于领域偏差，训练好的模型无法直接应用于新场景。在本文中，我们提出了一种新颖的半监督框架StyHighNet，用于在仅需要少量标记数据的新城市中准确估计单张航空图像的高度。其核心是将多源图像转换为统一风格，使未标记数据提供外观分布作为额外的监督信号。该框架主要包含三个子网络：（1）风格转换子网络将多源图像映射为统一风格分布图（USDM）；（2）高度回归子网络，其功能是从USDM预测高度图；（3）风格判别子网络，用于区分USDM的来源。其中，风格转换子网络肩负双重责任：一方面，它需要计算具有明显特征的USDM，以便高度回归子网络能够准确估计高度图。另一方面，USDM必须具有一致的分布以混淆风格判别子网络，从而实现领域自适应的目标。与以往方法不同，我们的风格分布函数是无监督学习得到的，因此具有更大的灵活性和更高的准确性。此外，当屏蔽风格判别子网络时，该框架也可用于监督学习。我们对两组公开数据Vaihingen和Potsdam进行了定性和定量评估。实验表明，该框架在监督学习和半监督学习模式下均取得了优异的性能。
