Xian Yan, Yu Hong, Wang Ye, Wang Guoyin
Chongqing Key Laboratory of Computational Intelligence, Chongqing University of Posts and Telecommunications, No.2 Chongwen Road, Chongqing, 400065, China.
National Center for Applied Mathematics in Chongqing, Chongqing Normal University, No. 37 Middle University Road, Chongqing, 401331, China.
Brain Inform. 2025 Mar 17;12(1):7. doi: 10.1186/s40708-025-00255-0.
Class incremental learning (CIL) is a specific scenario in incremental learning. It aims to continuously learn new classes from the data stream, which suffers from the challenge of catastrophic forgetting. Inspired by the human hippocampus, the CIL method for replaying episodic memory offers a promising solution. However, the limited buffer budget restricts the number of old class samples that can be stored, resulting in an imbalance between new and old class samples during each incremental learning stage. This imbalance adversely affects the mitigation of catastrophic forgetting. Therefore, we propose a novel CIL method based on multi-granularity balance strategy (MGBCIL), which is inspired by the three-way granular computing in human problem-solving. In order to mitigate the adverse effects of imbalances on catastrophic forgetting at fine-, medium-, and coarse-grained levels during training, MGBCIL introduces specific strategies across the batch, task, and decision stages. Specifically, a weighted cross-entropy loss function with a smoothing factor is proposed for batch processing. In the process of task updating and classification decision, contrastive learning with different anchor point settings is employed to promote local and global separation between new and old classes. Additionally, the knowledge distillation technology is used to preserve knowledge of the old classes. Experimental evaluations on CIFAR-10 and CIFAR-100 datasets show that MGBCIL outperforms other methods in most incremental settings. Specifically, when storing 3 exemplars on CIFAR-10 with Base2 Inc2 setting, the average accuracy is improved by up to 9.59% and the forgetting rate is reduced by up to 25.45%.
类别增量学习（CIL）是增量学习中的一种特定场景。它旨在从数据流中持续学习新类别，面临灾难性遗忘的挑战。受人类海马体启发，用于重放情景记忆的CIL方法提供了一个有前景的解决方案。然而，有限的缓冲区预算限制了可存储的旧类别样本数量，导致每个增量学习阶段新旧类别样本之间的不平衡。这种不平衡对减轻灾难性遗忘产生不利影响。因此，我们提出了一种基于多粒度平衡策略的新型CIL方法（MGBCIL），它受人类解决问题的三元粒度计算启发。为了在训练过程中减轻不平衡在细粒度、中粒度和粗粒度水平上对灾难性遗忘的不利影响，MGBCIL在批次、任务和决策阶段引入了特定策略。具体而言，针对批次处理提出了一种带有平滑因子的加权交叉熵损失函数。在任务更新和分类决策过程中，采用不同锚点设置的对比学习来促进新旧类别之间的局部和全局分离。此外，使用知识蒸馏技术来保留旧类别的知识。在CIFAR - 10和CIFAR - 100数据集上的实验评估表明，MGBCIL在大多数增量设置下优于其他方法。具体来说，在CIFAR - 10上使用Base2 Inc2设置存储3个样本时，平均准确率提高了高达9.59%，遗忘率降低了高达25.45%。
