Wei Qinglai, Zhu Liao, Song Ruizhuo, Zhang Pinjia, Liu Derong, Xiao Jun
IEEE Trans Neural Netw Learn Syst. 2022 Feb;33(2):879-892. doi: 10.1109/TNNLS.2020.3030127. Epub 2022 Feb 3.
In this article, an online adaptive optimal control algorithm based on adaptive dynamic programming is developed to solve the multiplayer nonzero-sum game (MP-NZSG) for discrete-time unknown nonlinear systems. First, a model-free coupled globalized dual-heuristic dynamic programming (GDHP) structure is designed to solve the MP-NZSG problem, in which there is no model network or identifier. Second, in order to relax the requirement of systems dynamics, an online adaptive learning algorithm is developed to solve the Hamilton-Jacobi equation using the system states of two adjacent time steps. Third, a series of critic networks and action networks are used to approximate value functions and optimal policies for all players. All the neural network (NN) weights are updated online based on real-time system states. Fourth, the uniformly ultimate boundedness analysis of the NN approximation errors is proved based on the Lyapunov approach. Finally, simulation results are given to demonstrate the effectiveness of the developed scheme.
在本文中，开发了一种基于自适应动态规划的在线自适应最优控制算法，以解决离散时间未知非线性系统的多人非零和博弈（MP-NZSG）问题。首先，设计了一种无模型耦合全局双启发式动态规划（GDHP）结构来解决MP-NZSG问题，其中不存在模型网络或标识符。其次，为了放宽对系统动力学的要求，开发了一种在线自适应学习算法，利用两个相邻时间步的系统状态来求解哈密顿-雅可比方程。第三，使用一系列评论家网络和动作网络来逼近所有参与者的值函数和最优策略。所有神经网络（NN）权重都基于实时系统状态进行在线更新。第四，基于李雅普诺夫方法证明了NN逼近误差的一致最终有界性分析。最后，给出了仿真结果以证明所开发方案的有效性。
