Department of Electronics and Communication Engineering, Khulna University of Engineering and Technology, Khulna-9203, Bangladesh.
Int J Neural Syst. 2011 Oct;21(5):427-41. doi: 10.1142/S0129065711002936.
This paper presents a pruning method for artificial neural networks (ANNs) based on the 'Lempel-Ziv complexity' (LZC) measure. We call this method the 'silent pruning algorithm' (SPA). The term 'silent' is used in the sense that SPA prunes ANNs without causing much disturbance during the network training. SPA prunes hidden units during the training process according to their ranks computed from LZC. LZC extracts the number of unique patterns in a time sequence obtained from the output of a hidden unit and a smaller value of LZC indicates higher redundancy of a hidden unit. SPA has a great resemblance to biological brains since it encourages higher complexity during the training process. SPA is similar to, yet different from, existing pruning algorithms. The algorithm has been tested on a number of challenging benchmark problems in machine learning, including cancer, diabetes, heart, card, iris, glass, thyroid, and hepatitis problems. We compared SPA with other pruning algorithms and we found that SPA is better than the 'random deletion algorithm' (RDA) which prunes hidden units randomly. Our experimental results show that SPA can simplify ANNs with good generalization ability.
本文提出了一种基于“Lempel-Ziv 复杂度”（LZC）度量的人工神经网络（ANNs）剪枝方法。我们称这种方法为“静默剪枝算法”（SPA）。“静默”一词是指 SPA 在不引起网络训练过程中过多干扰的情况下对 ANNs 进行剪枝。SPA 根据 LZC 计算的排名在训练过程中修剪隐藏单元。LZC 从隐藏单元的输出中提取时间序列中的独特模式数量，较小的 LZC 值表示隐藏单元的冗余度更高。SPA 与生物大脑非常相似，因为它在训练过程中鼓励更高的复杂性。SPA 类似于现有的剪枝算法，但又有所不同。该算法已经在机器学习中的一些具有挑战性的基准问题上进行了测试，包括癌症、糖尿病、心脏、卡片、虹膜、玻璃、甲状腺和肝炎问题。我们将 SPA 与其他剪枝算法进行了比较，发现 SPA 优于随机删除算法（RDA），后者随机修剪隐藏单元。我们的实验结果表明，SPA 可以简化具有良好泛化能力的 ANNs。
