Hong Wenjing, Li Guiying, Liu Shengcai, Yang Peng, Tang Ke
Guangdong Provincial Key Laboratory of Brain-Inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China.
Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology, Shenzhen 518055, China.
Fundam Res. 2022 Aug 9;4(4):941-950. doi: 10.1016/j.fmre.2022.07.013. eCollection 2024 Jul.
Neural network pruning is a popular approach to reducing the computational complexity of deep neural networks. In recent years, as growing evidence shows that conventional network pruning methods employ inappropriate proxy metrics, and as new types of hardware become increasingly available, hardware-aware network pruning that incorporates hardware characteristics in the loop of network pruning has gained growing attention. Both network accuracy and hardware efficiency (latency, memory consumption, etc.) are critical objectives to the success of network pruning, but the conflict between the multiple objectives makes it impossible to find a single optimal solution. Previous studies mostly convert the hardware-aware network pruning to optimization problems with a single objective. In this paper, we propose to solve the hardware-aware network pruning problem with Multi-Objective Evolutionary Algorithms (MOEAs). Specifically, we formulate the problem as a multi-objective optimization problem, and propose a novel memetic MOEA, namely HAMP, that combines an efficient portfolio-based selection and a surrogate-assisted local search, to solve it. Empirical studies demonstrate the potential of MOEAs in providing simultaneously a set of alternative solutions and the superiority of HAMP compared to the state-of-the-art hardware-aware network pruning method.
神经网络剪枝是一种降低深度神经网络计算复杂度的常用方法。近年来，越来越多的证据表明传统的网络剪枝方法使用了不合适的代理指标，并且随着新型硬件的日益普及，在网络剪枝过程中纳入硬件特性的硬件感知网络剪枝受到了越来越多的关注。网络准确性和硬件效率（延迟、内存消耗等）都是网络剪枝成功的关键目标，但多个目标之间的冲突使得无法找到单一的最优解。先前的研究大多将硬件感知网络剪枝转化为单目标优化问题。在本文中，我们提议使用多目标进化算法（MOEA）来解决硬件感知网络剪枝问题。具体而言，我们将该问题表述为一个多目标优化问题，并提出了一种新颖的混合多目标进化算法，即HAMP，它结合了基于有效投资组合的选择和代理辅助局部搜索来解决该问题。实证研究证明了多目标进化算法在同时提供一组替代解决方案方面的潜力，以及HAMP相对于当前最先进的硬件感知网络剪枝方法的优越性。
