Key Laboratory of Maritime Dynamic Simulation and Control of Ministry of Transportation, Dalian Maritime University, Dalian 116026, China.
Marine Engineering College, Dalian Maritime University, Dalian 116026, China.
Sensors (Basel). 2019 Sep 19;19(18):4055. doi: 10.3390/s19184055.
This research focuses on the adaptive navigation of maritime autonomous surface ships (MASSs) in an uncertain environment. To achieve intelligent obstacle avoidance of MASSs in a port, an autonomous navigation decision-making model based on hierarchical deep reinforcement learning is proposed. The model is mainly composed of two layers: the scene division layer and an autonomous navigation decision-making layer. The scene division layer mainly quantifies the sub-scenarios according to the International Regulations for Preventing Collisions at Sea (COLREG). This research divides the navigational situation of a ship into entities and attributes based on the ontology model and Protégé language. In the decision-making layer, we designed a deep Q-learning algorithm utilizing the environmental model, ship motion space, reward function, and search strategy to learn the environmental state in a quantized sub-scenario to train the navigation strategy. Finally, two sets of verification experiments of the deep reinforcement learning (DRL) and improved DRL algorithms were designed with Rizhao port as a study case. Moreover, the experimental data were analyzed in terms of the convergence trend, iterative path, and collision avoidance effect. The results indicate that the improved DRL algorithm could effectively improve the navigation safety and collision avoidance.
本研究聚焦于在不确定环境下的海上自主水面船舶（MASS）的自适应导航。为实现港口 MASS 的智能避碰，提出了一种基于分层深度强化学习的自主导航决策模型。该模型主要由两层组成：场景划分层和自主导航决策层。场景划分层主要根据《国际海上避碰规则》（COLREG）对子场景进行量化。本研究基于本体模型和 Protege 语言，根据航海情景的实体和属性对航海情景进行划分。在决策层，我们利用环境模型、船舶运动空间、奖励函数和搜索策略设计了一个深度 Q 学习算法，以量化的子场景中的环境状态进行学习，以训练导航策略。最后，以日照港为研究案例，设计了两组深度强化学习（DRL）和改进的 DRL 算法的验证实验，并从收敛趋势、迭代路径和避碰效果等方面对实验数据进行了分析。结果表明，改进的 DRL 算法能够有效提高导航安全性和避碰效果。
