Suppr超能文献

IESSP:基于信息提取的深度神经网络稀疏条纹剪枝方法

IESSP: Information Extraction-Based Sparse Stripe Pruning Method for Deep Neural Networks.

作者信息

Liu Jingjing, Huang Lingjin, Feng Manlong, Guo Aiying, Yin Luqiao, Zhang Jianhua

机构信息

Shanghai Key Laboratory of Chips and Systems for Intelligent Connected Vehicle, School of Microelectronics, Shanghai University, Shanghai 200444, China.

出版信息

Sensors (Basel). 2025 Apr 3;25(7):2261. doi: 10.3390/s25072261.

Abstract

Network pruning is a deep learning model compression technique aimed at reducing model storage requirements and decreasing computational resource consumption. However, mainstream pruning techniques often encounter challenges such as limited precision in feature selection and a diminished feature extraction capability. To address these issues, we propose an information extraction-based sparse stripe pruning (IESSP) method. This method introduces an information extraction module (IEM), which enhances stripe selection through a mask-based mechanism, promoting inter-layer interactions and directing the network's focus toward key features. In addition, we design a novel loss function that links output loss to stripe selection, enabling an effective balance between accuracy and efficiency. This loss function also supports the adaptive optimization of stripe sparsity during training. Experimental results on benchmark datasets demonstrate that the proposed method outperforms existing techniques. Specifically, when applied to prune the VGG-16 model on the CIFAR-10 dataset, the proposed method achieves a 0.29% improvement in accuracy while reducing FLOPs by 75.88% compared to the baseline.

摘要

网络剪枝是一种深度学习模型压缩技术,旨在降低模型存储需求并减少计算资源消耗。然而,主流剪枝技术常常面临诸如特征选择精度有限和特征提取能力下降等挑战。为了解决这些问题,我们提出了一种基于信息提取的稀疏条纹剪枝(IESSP)方法。该方法引入了一个信息提取模块(IEM),它通过基于掩码的机制增强条纹选择,促进层间交互并引导网络关注关键特征。此外,我们设计了一种新颖的损失函数,将输出损失与条纹选择联系起来,从而在准确性和效率之间实现有效平衡。该损失函数还支持在训练期间对条纹稀疏性进行自适应优化。在基准数据集上的实验结果表明,所提出的方法优于现有技术。具体而言,当应用于在CIFAR-10数据集上剪枝VGG-16模型时,与基线相比,所提出的方法在准确率上提高了0.29%,同时将浮点运算次数减少了75.88%。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/ee47/11991360/f24b3be7cd17/sensors-25-02261-g001.jpg

文献检索

告别复杂PubMed语法,用中文像聊天一样搜索,搜遍4000万医学文献。AI智能推荐,让科研检索更轻松。

立即免费搜索

文件翻译

保留排版,准确专业,支持PDF/Word/PPT等文件格式,支持 12+语言互译。

免费翻译文档

深度研究

AI帮你快速写综述,25分钟生成高质量综述,智能提取关键信息,辅助科研写作。

立即免费体验