Leite Arnø Mikkel, Godhavn John-Morten, Aamo Ole Morten
Department of Engineering Cybernetics, Norwegian University of Science and Technology, Trondheim 7491, Norway.
Equinor Research Center, Ranheim 7053, Norway.
MethodsX. 2021 Nov 12;8:101571. doi: 10.1016/j.mex.2021.101571. eCollection 2021.
In the streaming learning setting, an agent is presented with a data stream on which to learn from in an online fashion. A common problem is catastrophic forgetting of old knowledge due to updates to the model. Mitigating catastrophic forgetting has received a lot of attention, and a variety of methods exist to solve this problem. In this paper, we present a divided and prioritized experience replay approach for streaming regression, in which relevant observations are retained in the replay, and extra focus is added to poorly estimated observations through prioritization. Using a real-world dataset, the method is compared to the standard sliding window approach. A statistical power analysis is performed, showing how our approach improves performance on rare, important events at a trade-off in performance for more common observations. Close inspections of the dataset are provided, with emphasis on areas where the standard approach fails. A rephrasing of the problem to a binary classification problem is performed to separate common and rare, important events. These results provide an added perspective regarding the improvement made on rare events.••.
在流式学习环境中，智能体以在线方式接收一个用于从中学习的数据流。一个常见问题是由于模型更新导致对旧知识的灾难性遗忘。减轻灾难性遗忘受到了很多关注，并且存在多种方法来解决这个问题。在本文中，我们提出了一种用于流式回归的划分和优先经验回放方法，其中相关观测被保留在回放中，并且通过优先级对估计不佳的观测给予额外关注。使用一个真实世界数据集，将该方法与标准滑动窗口方法进行比较。进行了统计功效分析，展示了我们的方法如何在对更常见观测的性能进行权衡的情况下，提高对罕见重要事件的性能。提供了对数据集的详细检查，重点关注标准方法失败的区域。将问题重新表述为一个二元分类问题，以分离常见和罕见的重要事件。这些结果为关于在罕见事件上所做改进提供了一个额外的视角。••.
