Shin Jungwoo, Kim HyunJin
School of Electronics and Electrical Engineering, Dankook University, Yongin, South Korea.
PeerJ Comput Sci. 2022 Jan 3;8:e842. doi: 10.7717/peerj-cs.842. eCollection 2022.
In this study, we present a novel performance-enhancing binarized neural network model called PresB-Net: Parametric Binarized Neural Network. A binarized neural network (BNN) model can achieve fast output computation with low hardware costs by using binarized weights and features. However, performance degradation is the most critical problem in BNN models. Our PresB-Net combines several state-of-the-art BNN structures including the learnable activation with additional trainable parameters and shuffled grouped convolution. Notably, we propose a new normalization approach, which reduces the imbalance between the shuffled groups occurring in shuffled grouped convolutions. Besides, the proposed normalization approach helps gradient convergence so that the unstableness of the learning can be amortized when applying the learnable activation. Our novel BNN model enhances the classification performance compared with other existing BNN models. Notably, the proposed PresB-Net-18 achieves 73.84% Top-1 inference accuracy for the CIFAR-100 dataset, outperforming other existing counterparts.
在本研究中，我们提出了一种名为PresB-Net：参数化二值神经网络的新型性能增强二值神经网络模型。二值神经网络（BNN）模型通过使用二值化权重和特征，可以以较低的硬件成本实现快速的输出计算。然而，性能下降是BNN模型中最关键的问题。我们的PresB-Net结合了几种先进的BNN结构，包括具有额外可训练参数的可学习激活和随机分组卷积。值得注意的是，我们提出了一种新的归一化方法，该方法减少了随机分组卷积中出现的随机组之间的不平衡。此外，所提出的归一化方法有助于梯度收敛，从而在应用可学习激活时可以消除学习的不稳定性。与其他现有的BNN模型相比，我们的新型BNN模型提高了分类性能。值得注意的是，所提出的PresB-Net-18在CIFAR-100数据集上实现了73.84%的Top-1推理准确率，优于其他现有的同类模型。
