IEEE Trans Neural Netw Learn Syst. 2015 Mar;26(3):472-85. doi: 10.1109/TNNLS.2014.2315622.
The stochastic optimal control of nonlinear networked control systems (NNCSs) using neuro-dynamic programming (NDP) over a finite time horizon is a challenging problem due to terminal constraints, system uncertainties, and unknown network imperfections, such as network-induced delays and packet losses. Since the traditional iteration or time-based infinite horizon NDP schemes are unsuitable for NNCS with terminal constraints, a novel time-based NDP scheme is developed to solve finite horizon optimal control of NNCS by mitigating the above-mentioned challenges. First, an online neural network (NN) identifier is introduced to approximate the control coefficient matrix that is subsequently utilized in conjunction with the critic and actor NNs to determine a time-based stochastic optimal control input over finite horizon in a forward-in-time and online manner. Eventually, Lyapunov theory is used to show that all closed-loop signals and NN weights are uniformly ultimately bounded with ultimate bounds being a function of initial conditions and final time. Moreover, the approximated control input converges close to optimal value within finite time. The simulation results are included to show the effectiveness of the proposed scheme.
基于神经动态规划（NDP）的非线性网络控制系统（NNCSs）在有限时间域内的随机最优控制是一个具有挑战性的问题，因为存在终端约束、系统不确定性和未知的网络不完善性，如网络诱导延迟和数据包丢失。由于传统的基于迭代或基于时间的无限时域 NDP 方案不适合具有终端约束的 NNCS，因此开发了一种新的基于时间的 NDP 方案，通过减轻上述挑战来解决 NNCS 的有限时域最优控制问题。首先，引入在线神经网络（NN）标识符来近似控制系数矩阵，然后与评论家 NN 和动作家 NN 结合使用，以在线和向前的方式确定有限时域的基于时间的随机最优控制输入。最终，利用 Lyapunov 理论证明了所有闭环信号和 NN 权重都是一致有界的，最终界是初始条件和最终时间的函数。此外，近似控制输入在有限时间内收敛到接近最优值。仿真结果表明了所提出方案的有效性。
