Department of Electronic Engineering, University of Seville, 41009 Seville, Spain.
Sensors (Basel). 2021 Apr 19;21(8):2862. doi: 10.3390/s21082862.
The monitoring of water resources using Autonomous Surface Vehicles with water-quality sensors has been a recent approach due to the advances in unmanned transportation technology. The Ypacaraí Lake, the biggest water resource in Paraguay, suffers from a major contamination problem because of cyanobacteria blooms. In order to supervise the blooms using these on-board sensor modules, a Non-Homogeneous Patrolling Problem (a NP-hard problem) must be solved in a feasible amount of time. A dimensionality study is addressed to compare the most common methodologies, Evolutionary Algorithm and Deep Reinforcement Learning, in different map scales and fleet sizes with changes in the environmental conditions. The results determined that Deep Q-Learning overcomes the evolutionary method in terms of sample-efficiency by 50-70% in higher resolutions. Furthermore, it reacts better than the Evolutionary Algorithm in high space-state actions. In contrast, the evolutionary approach shows a better efficiency in lower resolutions and needs fewer parameters to synthesize robust solutions. This study reveals that Deep Q-learning approaches exceed in efficiency for the Non-Homogeneous Patrolling Problem but with many hyper-parameters involved in the stability and convergence.
利用带有水质传感器的自主水面车辆监测水资源是最近的一种方法，因为无人运输技术取得了进步。亚帕克拉伊湖是巴拉圭最大的水资源，但由于蓝藻水华，它面临着严重的污染问题。为了使用这些机载传感器模块来监测水华，必须在可行的时间内解决非均匀巡逻问题（一个 NP 难问题）。本研究进行了维度研究，以比较最常见的方法，进化算法和深度强化学习，在不同的地图比例和船队规模与环境条件的变化。结果表明，深度 Q 学习在样本效率方面比进化方法提高了 50-70%，在更高的分辨率下。此外，它在高空间状态动作中的反应优于进化算法。相比之下，进化方法在较低的分辨率下显示出更好的效率，并且需要较少的参数来综合鲁棒的解决方案。这项研究表明，深度 Q 学习方法在非均匀巡逻问题上的效率更高，但在稳定性和收敛性方面涉及到许多超参数。
