Wang Xiaoyu, Huang Yong
Clemson University, Clemson, SC 29630 USA.
IEEE Trans Neural Netw. 2011 Apr;22(4):588-600. doi: 10.1109/TNN.2011.2109737. Epub 2011 Mar 10.
Recurrent neural network (RNN) has emerged as a promising tool in modeling nonlinear dynamical systems, but the training convergence is still of concern. This paper aims to develop an effective extended Kalman filter-based RNN training approach with a controllable training convergence. The training convergence problem during extended Kalman filter-based RNN training has been proposed and studied by adapting two artificial training noise parameters: the covariance of measurement noise (R) and the covariance of process noise (Q) of Kalman filter. The R and Q adaption laws have been developed using the Lyapunov method and the maximum likelihood method, respectively. The effectiveness of the proposed adaption laws has been tested using a nonlinear dynamical benchmark system and further applied in cutting tool wear modeling. The results show that the R adaption law can effectively avoid the divergence problem and ensure the training convergence, whereas the Q adaption law helps improve the training convergence speed.
递归神经网络（RNN）已成为一种在非线性动力系统建模中很有前景的工具，但训练收敛性仍然令人关注。本文旨在开发一种基于扩展卡尔曼滤波器的有效RNN训练方法，使其具有可控的训练收敛性。通过调整两个人工训练噪声参数：卡尔曼滤波器的测量噪声协方差（R）和过程噪声协方差（Q），提出并研究了基于扩展卡尔曼滤波器的RNN训练过程中的训练收敛问题。分别使用李雅普诺夫方法和最大似然方法开发了R和Q的自适应律。利用一个非线性动力基准系统测试了所提出的自适应律的有效性，并进一步应用于刀具磨损建模。结果表明，R自适应律可以有效避免发散问题并确保训练收敛，而Q自适应律有助于提高训练收敛速度。
