Università degli Studi di Torino, corso Svizzera 185, Torino, Italy; LTCI, Télécom Paris, Institut Polytechnique de Paris, France.
Università degli Studi di Torino, corso Svizzera 185, Torino, Italy.
Neural Netw. 2022 Feb;146:230-237. doi: 10.1016/j.neunet.2021.11.029. Epub 2021 Dec 2.
LOBSTER (LOss-Based SensiTivity rEgulaRization) is a method for training neural networks having a sparse topology. Let the sensitivity of a network parameter be the variation of the loss function with respect to the variation of the parameter. Parameters with low sensitivity, i.e. having little impact on the loss when perturbed, are shrunk and then pruned to sparsify the network. Our method allows to train a network from scratch, i.e. without preliminary learning or rewinding. Experiments on multiple architectures and datasets show competitive compression ratios with minimal computational overhead.
LOBSTER（基于损失的敏感性正则化）是一种训练具有稀疏拓扑结构的神经网络的方法。让网络参数的敏感性是损失函数随参数变化的变化。敏感性低的参数，即当受到干扰时对损失的影响较小，被收缩然后修剪以稀疏化网络。我们的方法允许从头开始训练网络，即无需预先学习或回滚。在多个架构和数据集上的实验表明，在最小计算开销的情况下具有竞争力的压缩比。
