Patra Arijit, Cai Yifan, Chatelain Pierre, Sharma Harshita, Drukker Lior, Papageorghiou Aris T, Noble J Alison
University of Oxford, Oxford, Oxfordshire OX3 7DQ, UK.
Simpl Med Ultrasound (2021). 2021 Sep 21;12967:14-24. doi: 10.1007/978-3-030-87583-1_2.
Deep networks have been shown to achieve impressive accuracy for some medical image analysis tasks where large datasets and annotations are available. However, tasks involving learning over new sets of classes arriving over extended time is a different and difficult challenge due to the tendency of reduction in performance over old classes while adapting to new ones. Controlling such a 'forgetting' is vital for deployed algorithms to evolve with new arrivals of data incrementally. Usually, incremental learning approaches rely on expert knowledge in the form of manual annotations or active feedback. In this paper, we explore the role that other forms of expert knowledge might play in making deep networks in medical image analysis immune to forgetting over extended time. We introduce a novel framework for mitigation of this forgetting effect in deep networks considering the case of combining ultrasound video with point-of-gaze tracked for expert sonographers during model training. This is used along with a novel weighted distillation strategy to reduce the propagation of effects due to class imbalance.
深度网络已被证明在某些有大量数据集和注释的医学图像分析任务中能取得令人印象深刻的准确率。然而，由于在适应新类别时旧类别性能会下降，涉及对长时间内新出现的类别集进行学习的任务是一个不同且困难的挑战。控制这种“遗忘”对于已部署的算法随着新数据的不断到来而逐步发展至关重要。通常，增量学习方法依赖于以手动注释或主动反馈形式存在的专家知识。在本文中，我们探讨了其他形式的专家知识在使医学图像分析中的深度网络在长时间内免受遗忘影响方面可能发挥的作用。考虑到在模型训练期间将超声视频与专家超声医师的注视点跟踪相结合的情况，我们引入了一个减轻深度网络中这种遗忘效应的新颖框架。这与一种新颖的加权蒸馏策略一起使用，以减少由于类别不平衡导致的影响传播。
