Yam J F, Chow T S
City Univ. of Hong Kong, Kowloon.
IEEE Trans Neural Netw. 1997;8(3):806-10. doi: 10.1109/72.572119.
An extended least squares-based algorithm for feedforward networks is proposed. The weights connecting the last hidden and output layers are first evaluated by least squares algorithm. The weights between input and hidden layers are then evaluated using the modified gradient descent algorithms. This arrangement eliminates the stalling problem experienced by the pure least squares type algorithms; however, still maintains the characteristic of fast convergence. In the investigated problems, the total number of FLOPS required for the networks to converge using the proposed training algorithm are only 0.221%-16.0% of that using the Levenberg-Marquardt algorithm. The number of floating point operations per iteration of the proposed algorithm are only 1.517-3.521 times of that of the standard backpropagation algorithm.
提出了一种基于扩展最小二乘法的前馈网络算法。首先通过最小二乘法评估连接最后一个隐藏层和输出层的权重。然后使用改进的梯度下降算法评估输入层和隐藏层之间的权重。这种安排消除了纯最小二乘类型算法所经历的停滞问题；然而，仍然保持快速收敛的特性。在所研究的问题中，使用所提出的训练算法使网络收敛所需的浮点运算总数仅为使用Levenberg-Marquardt算法时的0.221%-16.0%。所提出算法每次迭代的浮点运算次数仅为标准反向传播算法的1.517-3.521倍。
