School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China.
Lianyungang E-Port Information Development Co. Ltd., Lianyungang 222042, China.
Sensors (Basel). 2018 Oct 25;18(11):3627. doi: 10.3390/s18113627.
Anomaly detection aims to separate anomalous pixels from the background, and has become an important application of remotely sensed hyperspectral image processing. Anomaly detection methods based on low-rank and sparse representation (LRASR) can accurately detect anomalous pixels. However, with the significant volume increase of hyperspectral image repositories, such techniques consume a significant amount of time (mainly due to the massive amount of matrix computations involved). In this paper, we propose a novel distributed parallel algorithm (DPA) by redesigning key operators of LRASR in terms of MapReduce model to accelerate LRASR on cloud computing architectures. Independent computation operators are explored and executed in parallel on Spark. Specifically, we reconstitute the hyperspectral images in an appropriate format for efficient DPA processing, design the optimized storage strategy, and develop a pre-merge mechanism to reduce data transmission. Besides, a repartitioning policy is also proposed to improve DPA's efficiency. Our experimental results demonstrate that the newly developed DPA achieves very high speedups when accelerating LRASR, in addition to maintaining similar accuracies. Moreover, our proposed DPA is shown to be scalable with the number of computing nodes and capable of processing big hyperspectral images involving massive amounts of data.
异常检测旨在将异常像素与背景分离，已成为遥感高光谱图像处理的一个重要应用。基于低秩和稀疏表示 (LRASR) 的异常检测方法可以准确地检测异常像素。然而，随着高光谱图像库的显著增长，此类技术需要消耗大量的时间（主要是由于涉及大量矩阵计算）。在本文中，我们通过重新设计基于 MapReduce 模型的 LRASR 的关键操作符，提出了一种新的分布式并行算法 (DPA)，以加速云计算架构上的 LRASR。在 Spark 上探索并并行执行独立的计算操作符。具体来说，我们以高效 DPA 处理的适当格式重新构成高光谱图像，设计优化的存储策略，并开发预合并机制以减少数据传输。此外，还提出了一种重新分区策略以提高 DPA 的效率。实验结果表明，新开发的 DPA 在加速 LRASR 时实现了非常高的加速比，同时保持了相似的准确性。此外，我们提出的 DPA 具有可扩展性，可以处理涉及大量数据的大型高光谱图像。
