Rehman Sadaqat Ur, Tu Shanshan, Rehman Obaid Ur, Huang Yongfeng, Magurawalage Chathura M Sarathchandra, Chang Chin-Chen
Department of Electronic Engineering, Tsinghua University, Beijing 100084, China.
Faculty of Information Technology, Beijing University of Technology, Beijing 100022, China.
Entropy (Basel). 2018 Apr 17;20(4):290. doi: 10.3390/e20040290.
The convolution neural network (CNN) has achieved state-of-the-art performance in many computer vision applications e.g., classification, recognition, detection, etc. However, the global optimization of CNN training is still a problem. Fast classification and training play a key role in the development of the CNN. We hypothesize that the smoother and optimized the training of a CNN goes, the more efficient the end result becomes. Therefore, in this paper, we implement a modified resilient backpropagation (MRPROP) algorithm to improve the convergence and efficiency of CNN training. Particularly, a tolerant band is introduced to avoid network overtraining, which is incorporated with the global best concept for weight updating criteria to allow the training algorithm of the CNN to optimize its weights more swiftly and precisely. For comparison, we present and analyze four different training algorithms for CNN along with MRPROP, i.e., resilient backpropagation (RPROP), Levenberg-Marquardt (LM), conjugate gradient (CG), and gradient descent with momentum (GDM). Experimental results showcase the merit of the proposed approach on a public face and skin dataset.
卷积神经网络（CNN）在许多计算机视觉应用中，如分类、识别、检测等，都取得了领先的性能。然而，CNN训练的全局优化仍然是一个问题。快速分类和训练在CNN的发展中起着关键作用。我们假设，CNN训练过程越平滑、优化，最终结果就越高效。因此，在本文中，我们实现了一种改进的弹性反向传播（MRPROP）算法，以提高CNN训练的收敛性和效率。特别是，引入了一个容忍带以避免网络过度训练，并将其与用于权重更新标准的全局最优概念相结合，以使CNN的训练算法能够更快速、精确地优化其权重。为了进行比较，我们展示并分析了与MRPROP一起的四种不同的CNN训练算法，即弹性反向传播（RPROP）、列文伯格-马夸尔特（LM）、共轭梯度（CG）和带动量的梯度下降（GDM）。实验结果展示了所提出方法在一个公开的面部和皮肤数据集上的优点。
