Berg Henrik Stokland, Menges Daniel, Tengesdal Trym, Rasheed Adil
Department of Engineering Cybernetics, Norwegian University of Science and Technology, Trondheim, Norway.
Department of Mathematics and Cybernetics, SINTEF Digital, Trondheim, Norway.
Sci Rep. 2025 Mar 18;15(1):9344. doi: 10.1038/s41598-025-93635-9.
Current control systems for autonomous surface vessels (ASVs) often disregard model uncertainties and the need to adapt dynamically to varying model parameters. This limitation hinders their ability to ensure reliable performance under complex and frequently changing maritime conditions, highlighting the need for more adaptive and robust approaches. Therefore, this study introduces an innovative approach that integrates deep reinforcement learning (DRL) with nonlinear model predictive control (NMPC) to optimize the control performance and model parameters of ASVs. The primary objective is to ensure that the digital twin of the ASV remains continuously synchronized with its physical counterpart, thereby enhancing the accuracy, reliability, and adaptability of the digital twin in representing the vessel under complex and dynamic maritime conditions. Leveraging the capabilities of digital twins, agents can be trained in safety-critical applications within a risk-free virtual environment, minimizing the hazards associated with real-world experimentation. The DRL framework optimizes NMPC by tuning its parameters for peak performance and identifying unknown model parameters in real-time, ensuring precise and dependable vessel control. Extensive simulations confirm the effectiveness of this approach in improving the safety, efficiency, and reliability of ASVs. The proposed methods address critical challenges in ASV control by enhancing reliability and adaptability under dynamic conditions, providing a foundation for future advancements in autonomous maritime navigation and control system development.
当前的自主水面舰艇（ASV）控制系统常常忽视模型的不确定性以及动态适应变化模型参数的需求。这一局限性阻碍了它们在复杂且频繁变化的海上条件下确保可靠性能的能力，凸显了对更具适应性和鲁棒性方法的需求。因此，本研究引入了一种创新方法，将深度强化学习（DRL）与非线性模型预测控制（NMPC）相结合，以优化ASV的控制性能和模型参数。主要目标是确保ASV的数字孪生与其物理实体持续同步，从而提高数字孪生在复杂动态海上条件下表示船舶时的准确性、可靠性和适应性。利用数字孪生的能力，可以在无风险的虚拟环境中对关键安全应用中的智能体进行训练，将与实际实验相关的风险降至最低。DRL框架通过调整NMPC的参数以实现最佳性能并实时识别未知模型参数来优化NMPC，确保精确可靠的船舶控制。大量仿真证实了该方法在提高ASV的安全性、效率和可靠性方面的有效性。所提出的方法通过增强动态条件下的可靠性和适应性解决了ASV控制中的关键挑战，为自主海上导航和控制系统开发的未来进展奠定了基础。
