Indian Institute of Technology Jodhpur, India.
Indian Institute of Technology Roorkee, India.
Neural Netw. 2023 Aug;165:999-1009. doi: 10.1016/j.neunet.2023.06.043. Epub 2023 Jul 5.
In the few-shot class incremental learning (FSCIL) setting, new classes with few training examples become available incrementally, and deep learning models suffer from catastrophic forgetting of the previous classes when trained on new classes. Data augmentation techniques are generally used to increase the training data and improve the model performance. In this work, we demonstrate that differently augmented views of the same image obtained by applying data augmentations may not necessarily activate the same set of neurons in the model. Therefore, the information gained by a model regarding a class, when trained using data augmentation, may not necessarily be stored in the same set of neurons in the model. Consequently, during incremental training, even if some of the model weights that store the previously seen class information for a particular view get overwritten, the information of the previous classes for the other views may still remain intact in the other model weights. Therefore, the impact of catastrophic forgetting on the model predictions is different for different data augmentations used during training. Based on this, we present an Augmentation-based Prediction Rectification (APR) approach to reduce the impact of catastrophic forgetting in the FSCIL setting. APR can also augment other FSCIL approaches and significantly improve their performance. We also propose a novel feature synthesis module (FSM) for synthesizing features relevant to the previously seen classes without requiring training data from these classes. FSM outperforms other generative approaches in this setting. We experimentally show that our approach outperforms other methods on benchmark datasets.
在少样本增量学习(FSCIL)设置中,新的类别的训练样本很少,并且当在新类上训练时,深度学习模型会灾难性地忘记以前的类。通常使用数据增强技术来增加训练数据并提高模型性能。在这项工作中,我们证明了通过应用数据增强获得的同一图像的不同增强视图不一定会激活模型中的同一组神经元。因此,模型在使用数据增强进行训练时获得的关于一个类的信息,不一定存储在模型中的同一组神经元中。因此,在增量训练期间,即使存储特定视图的先前看到的类信息的一些模型权重被覆盖,其他视图的先前类的信息在其他模型权重中仍可能保持完整。因此,在训练期间使用的不同数据增强对模型预测的灾难性遗忘的影响是不同的。基于此,我们提出了一种基于增强的预测校正(APR)方法来减少 FSCIL 设置中的灾难性遗忘的影响。APR 还可以增强其他 FSCIL 方法,并显著提高它们的性能。我们还提出了一种新的特征合成模块(FSM),用于在不要求这些类别的训练数据的情况下合成与以前看到的类相关的特征。在这种设置下,FSM 优于其他生成方法。我们通过实验表明,我们的方法在基准数据集上优于其他方法。