Wang Ting, Zhang Mingyang, Zhang Jianjun, Ng Wing W Y, Chen C L Philip
IEEE Trans Neural Netw Learn Syst. 2024 Feb;35(2):1681-1695. doi: 10.1109/TNNLS.2022.3184846. Epub 2024 Feb 5.
The training of the standard broad learning system (BLS) concerns the optimization of its output weights via the minimization of both training mean square error (MSE) and a penalty term. However, it degrades the generalization capability and robustness of BLS when facing complex and noisy environments, especially when small perturbations or noise appear in input data. Therefore, this work proposes a broad network based on localized stochastic sensitivity (BASS) algorithm to tackle the issue of noise or input perturbations from a local perturbation perspective. The localized stochastic sensitivity (LSS) prompts an increase in the network's noise robustness by considering unseen samples located within a Q -neighborhood of training samples, which enhances the generalization capability of BASS with respect to noisy and perturbed data. Then, three incremental learning algorithms are derived to update BASS quickly when new samples arrive or the network is deemed to be expanded, without retraining the entire model. Due to the inherent superiorities of the LSS, extensive experimental results on 13 benchmark datasets show that BASS yields better accuracies on various regression and classification problems. For instance, BASS uses fewer parameters (12.6 million) to yield 1% higher Top-1 accuracy in comparison to AlexNet (60 million) on the large-scale ImageNet (ILSVRC2012) dataset.
标准广义学习系统（BLS）的训练涉及通过最小化训练均方误差（MSE）和一个惩罚项来优化其输出权重。然而，当面对复杂且有噪声的环境时，尤其是当输入数据中出现小扰动或噪声时，它会降低BLS的泛化能力和鲁棒性。因此，这项工作提出了一种基于局部随机敏感性（BASS）算法的广义网络，从局部扰动的角度解决噪声或输入扰动问题。局部随机敏感性（LSS）通过考虑位于训练样本的Q邻域内的未见过的样本，促使网络的噪声鲁棒性增加，这增强了BASS对噪声和扰动数据的泛化能力。然后，推导了三种增量学习算法，以便在新样本到达或网络被认为需要扩展时快速更新BASS，而无需重新训练整个模型。由于LSS固有的优势，在13个基准数据集上的大量实验结果表明，BASS在各种回归和分类问题上产生了更好的准确率。例如，在大规模ImageNet（ILSVRC2012）数据集上，与AlexNet（6000万）相比，BASS使用更少的参数（1260万），Top-1准确率高出1%。
