Wei Qinglai, Liao Zehua, Yang Zhanyu, Li Benkai, Liu Derong
IEEE Trans Cybern. 2020 Dec;50(12):4958-4971. doi: 10.1109/TCYB.2019.2926631. Epub 2020 Dec 3.
A novel policy iteration algorithm, called the continuous-time time-varying (CTTV) policy iteration algorithm, is presented in this paper to obtain the optimal control laws for infinite horizon CTTV nonlinear systems. The adaptive dynamic programming (ADP) technique is utilized to obtain the iterative control laws for the optimization of the performance index function. The properties of the CTTV policy iteration algorithm are analyzed. Monotonicity, convergence, and optimality of the iterative value function have been analyzed, and the iterative value function can be proven to monotonically converge to the optimal solution of the Hamilton-Jacobi-Bellman (HJB) equation. Furthermore, the iterative control law is guaranteed to be admissible to stabilize the nonlinear systems. In the implementation of the presented CTTV policy algorithm, the approximate iterative control laws and iterative value function are obtained by neural networks. Finally, the numerical results are given to verify the effectiveness of the presented method.
本文提出了一种新颖的策略迭代算法，称为连续时间时变（CTTV）策略迭代算法，用于获得无限时域CTTV非线性系统的最优控制律。利用自适应动态规划（ADP）技术来获得用于性能指标函数优化的迭代控制律。分析了CTTV策略迭代算法的性质。对迭代值函数的单调性、收敛性和最优性进行了分析，并且可以证明迭代值函数单调收敛到汉密尔顿-雅可比-贝尔曼（HJB）方程的最优解。此外，保证迭代控制律可用于稳定非线性系统。在所提出的CTTV策略算法的实现中，通过神经网络获得近似迭代控制律和迭代值函数。最后，给出数值结果以验证所提方法的有效性。
