Castellano G, Fanelli A M, Pelillo M
CNR, Bari.
IEEE Trans Neural Netw. 1997;8(3):519-31. doi: 10.1109/72.572092.
The problem of determining the proper size of an artificial neural network is recognized to be crucial, especially for its practical implications in such important issues as learning and generalization. One popular approach for tackling this problem is commonly known as pruning and it consists of training a larger than necessary network and then removing unnecessary weights/nodes. In this paper, a new pruning method is developed, based on the idea of iteratively eliminating units and adjusting the remaining weights in such a way that the network performance does not worsen over the entire training set. The pruning problem is formulated in terms of solving a system of linear equations, and a very efficient conjugate gradient algorithm is used for solving it, in the least-squares sense. The algorithm also provides a simple criterion for choosing the units to be removed, which has proved to work well in practice. The results obtained over various test problems demonstrate the effectiveness of the proposed approach.
确定人工神经网络的合适规模问题被认为至关重要，特别是考虑到其在诸如学习和泛化等重要问题中的实际影响。一种解决该问题的常用方法通常称为剪枝，它包括训练一个规模大于所需的网络，然后去除不必要的权重/节点。本文基于迭代消除单元并调整剩余权重以使网络在整个训练集上性能不会变差的思想，开发了一种新的剪枝方法。剪枝问题被表述为求解一个线性方程组，并且在最小二乘意义下使用一种非常有效的共轭梯度算法来求解它。该算法还提供了一个选择要去除单元的简单准则，实践证明该准则效果良好。在各种测试问题上获得的结果证明了所提方法的有效性。
