Xiao Geyang, Zhang Huaguang
IEEE Trans Cybern. 2024 Mar;54(3):1639-1649. doi: 10.1109/TCYB.2022.3232599. Epub 2024 Feb 9.
This article is concerned with the convergence property and error bounds analysis of value iteration (VI) adaptive dynamic programming for continuous-time (CT) nonlinear systems. The size relationship between the total value function and the single integral step cost is described by assuming a contraction assumption. Then, the convergence property of VI is proved while the initial condition is an arbitrary positive semidefinite function. Moreover, the accumulated effects of approximation errors generated in each iteration are taken into consideration while using approximators to implement the algorithm. Based on the contraction assumption, the error bounds condition is proposed, which ensures the approximated iterative results converge to a neighborhood of the optimum, and the relation between the optimal solution and approximated iterative results is also derived. To make the contraction assumption more concrete, an estimation way is proposed to derive a conservative value of the assumption. Finally, three simulation cases are given to validate the theoretical results.
本文关注连续时间（CT）非线性系统的价值迭代（VI）自适应动态规划的收敛性和误差界分析。通过假设一个收缩假设来描述总值函数与单积分步成本之间的大小关系。然后，证明了当初始条件为任意半正定函数时VI的收敛性。此外，在使用逼近器实现算法时，考虑了每次迭代中产生的逼近误差的累积效应。基于收缩假设，提出了误差界条件，该条件确保逼近的迭代结果收敛到最优值的邻域，并推导了最优解与逼近迭代结果之间的关系。为了使收缩假设更具体，提出了一种估计方法来推导该假设的保守值。最后，给出了三个仿真案例来验证理论结果。
