IEEE Trans Cybern. 2020 Feb;50(2):452-464. doi: 10.1109/TCYB.2018.2873762. Epub 2018 Oct 18.
Recently, the high computational resource demands of convolutional neural networks (CNNs) have hindered a wide range of their applications. To solve this problem, many previous works attempted to reduce the redundant calculations during the evaluation of CNNs. However, these works mainly focused on either interspatial or interkernel redundancy. In this paper, we further accelerate existing CNNs by removing both types of redundancies. First, we convert interspatial redundancy into interkernel redundancy by decomposing one convolutional layer to one block that we design. Then, we adopt rank-selection and pruning methods to remove the interkernel redundancy. The rank-selection method, which considerably reduces manpower, contributes to determining the number of kernels to be pruned in the pruning method. We apply a layer-wise training algorithm rather than the traditional end-to-end training to overcome the difficulty of convergence. Finally, we fine-tune the entire network to achieve better performance. Our method is applied on three widely used datasets of an image classification task. We achieve better results in terms of accuracy and compression rate compared with previous state-of-the-art methods.
最近，卷积神经网络（CNN）的高计算资源需求阻碍了它们的广泛应用。为了解决这个问题，许多先前的工作试图减少 CNN 评估过程中的冗余计算。然而，这些工作主要集中在空间或内核冗余上。在本文中，我们通过去除这两种类型的冗余进一步加速现有的 CNN。首先，我们通过将一个卷积层分解为我们设计的一个块，将空间冗余转化为内核冗余。然后，我们采用秩选择和剪枝方法来去除内核冗余。秩选择方法大大减少了人力，有助于确定剪枝方法中要剪枝的核的数量。我们采用逐层训练算法而不是传统的端到端训练来克服收敛困难。最后，我们对整个网络进行微调以获得更好的性能。我们的方法应用于图像分类任务的三个广泛使用的数据集上。与先前的最先进方法相比，我们在准确性和压缩率方面取得了更好的结果。
