School of Information Science and Engineering, Chongqing Jiaotong University, Chongqing, China; School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China.
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China.
Neural Netw. 2023 Oct;167:706-714. doi: 10.1016/j.neunet.2023.08.063. Epub 2023 Sep 9.
Adversarial training is considered one of the most effective methods to improve the adversarial robustness of deep neural networks. Despite the success, it still suffers from unsatisfactory performance and overfitting. Considering the intrinsic mechanism of adversarial training, recent studies adopt the idea of curriculum learning to alleviate overfitting. However, this also introduces new issues, that is, lacking the quantitative criterion for attacks' strength and catastrophic forgetting. To mitigate such issues, we propose the self-paced adversarial training (SPAT), which explicitly builds the learning process of adversarial training based on adversarial examples of the whole dataset. Specifically, our model is first trained with "easy" adversarial examples, and then is continuously enhanced by gradually adding "complex" adversarial examples. This way strengthens the ability to fit "complex" adversarial examples while holding in mind "easy" adversarial samples. To balance adversarial examples between classes, we determine the difficulty of the adversarial examples locally in each class. Notably, this learning paradigm can also be incorporated into other advanced methods for further boosting adversarial robustness. Experimental results show the effectiveness of our proposed model against various attacks on widely-used benchmarks. Especially, on CIFAR100, SPAT provides a boost of 1.7% (relatively 5.4%) in robust accuracy on the PGD10 attack and 3.9% (relatively 7.2%) in natural accuracy for AWP.
对抗训练被认为是提高深度神经网络对抗鲁棒性的最有效方法之一。尽管取得了成功，但它仍然存在性能不佳和过拟合的问题。考虑到对抗训练的内在机制，最近的研究采用课程学习的思想来缓解过拟合。然而，这也引入了新的问题，即缺乏攻击强度的定量标准和灾难性遗忘。为了解决这些问题，我们提出了自步对抗训练（SPAT），它基于整个数据集的对抗样本来显式地构建对抗训练的学习过程。具体来说，我们的模型首先用“简单”的对抗样本来训练，然后通过逐渐添加“复杂”的对抗样本来不断增强。这种方式在记住“简单”对抗样本的同时，增强了对“复杂”对抗样本的适应能力。为了在类之间平衡对抗样例，我们在每个类中本地确定对抗样例的难度。值得注意的是，这种学习范式也可以被纳入其他先进的方法中，以进一步提高对抗鲁棒性。实验结果表明，我们提出的模型在广泛使用的基准上针对各种攻击都具有有效性。特别是在 CIFAR100 上，SPAT 在 PGD10 攻击下的鲁棒准确率提高了 1.7%（相对提高 5.4%），在 AWP 下的自然准确率提高了 3.9%（相对提高 7.2%）。
