Anand R, Mehrotra K, Mohan C K, Ranka S
Sch. of Comput. and Inf. Sci., Syracuse Univ., NY.
IEEE Trans Neural Netw. 1995;6(1):117-24. doi: 10.1109/72.363444.
The rate of convergence of net output error is very low when training feedforward neural networks for multiclass problems using the backpropagation algorithm. While backpropagation will reduce the Euclidean distance between the actual and desired output vectors, the differences between some of the components of these vectors increase in the first iteration. Furthermore, the magnitudes of subsequent weight changes in each iteration are very small, so that many iterations are required to compensate for the increased error in some components in the initial iterations. Our approach is to use a modular network architecture, reducing a K-class problem to a set of K two-class problems, with a separately trained network for each of the simpler problems. Speedups of one order of magnitude have been obtained experimentally, and in some cases convergence was possible using the modular approach but not using a nonmodular network.
当使用反向传播算法训练前馈神经网络来解决多类问题时，网络输出误差的收敛速度非常低。虽然反向传播会减小实际输出向量与期望输出向量之间的欧几里得距离，但在第一次迭代中，这些向量某些分量之间的差异会增大。此外，每次迭代中后续权重变化的幅度非常小，因此需要许多次迭代来补偿初始迭代中某些分量增加的误差。我们的方法是使用模块化网络架构，将一个K类问题简化为一组K个二类问题，为每个较简单的问题分别训练一个网络。通过实验已经获得了一个数量级的加速，并且在某些情况下，使用模块化方法可以实现收敛，而使用非模块化网络则无法收敛。
