Zhou G, Si J
Department of Electrical Engineering, Arizona State University, Tempe, AZ 85287-5706, USA.
Neural Comput. 1998 May 15;10(4):1031-45. doi: 10.1162/089976698300017610.
Most neural network applications rely on the fundamental approximation property of feedforward networks. Supervised learning is a means of implementing this approximate mapping. In a realistic problem setting, a mechanism is needed to devise this learning process based on available data, which encompasses choosing an appropriate set of parameters in order to avoid overfitting, using an efficient learning algorithm measured by computation and memory complexities, ensuring the accuracy of the training procedures as measured by the training error, and testing and cross-validation for generalization. We develop a comprehensive supervised learning algorithm to address these issues. The algorithm combines training and pruning into one procedure by utilizing a common observation of Jacobian rank deficiency in feedforward networks. The algorithm not only reduces the training time and overall complexity but also achieves training accuracy and generalization capabilities comparable to more standard approaches. Extensive simulation results are provided to demonstrate the effectiveness of the algorithm.
大多数神经网络应用依赖于前馈网络的基本逼近特性。监督学习是实现这种近似映射的一种手段。在实际问题设置中，需要一种机制来基于可用数据设计这种学习过程，这包括选择一组合适的参数以避免过拟合，使用由计算和内存复杂度衡量的高效学习算法，确保由训练误差衡量的训练过程的准确性，以及进行测试和交叉验证以实现泛化。我们开发了一种综合监督学习算法来解决这些问题。该算法通过利用前馈网络中雅可比矩阵秩亏缺的常见观察结果，将训练和剪枝合并为一个过程。该算法不仅减少了训练时间和总体复杂度，还实现了与更标准方法相当的训练准确性和泛化能力。提供了广泛的仿真结果来证明该算法的有效性。
