IEEE J Biomed Health Inform. 2020 Apr;24(4):1046-1058. doi: 10.1109/JBHI.2020.2973372. Epub 2020 Feb 12.
This paper proposes an ultrasound video interpretation algorithm that enables novel classes or instances to be added over time, without significantly compromising prediction abilities on prior representations. The motivating application is diagnostic fetal echocardiography analysis. Currently in clinical practice, recording full diagnostic fetal echocardiography is not common. Diagnostic videos are typically available in varying length and summarize a number of diagnostic sub-tasks of varying difficulty. Although large clinical datasets may be available at onset to build ultrasound image-based models for automatic image analysis, data may also become available over extended time to assist in algorithm refinement. To address this scenario, we propose to use an incremental learning approach to build a hierarchical network model that allows for a parallel inclusion of previously unseen anatomical classes without requiring prior data distributions. Super classes are obtained by coarse classification followed by fine classification to allow the model to self-organize anatomical structures in a sequence of categories through a modular architecture. We show that this approach can be adapted with new variable data distributions without significantly affecting previously learned representations. Two extreme situations of new data addition are considered; (1) when new class data is available over time with volume and distribution similar to prior available classes, and (2) when imbalanced datasets arrive over future time to be learned in a few-shot setting. In either case, availability of data from prior classes is not assumed. Evolution of the learning process is validated using incremental accuracies of fine classification over novel classes and compared to results from an end-to-end transfer learning-derived model fine-tuned on a clinical dataset annotated by experienced sonographers. The modularization of subsequent learning reduces the depreciation in future accuracies over old tasks from 6.75% to 1.10% using balanced increments. The depreciation is reduced from 6.95% to 1.89% with imbalanced data distributions in future increments, while retaining competitive classification accuracies in new additions of fine classes with parameter operations in the same order of magnitude in all stages in both cases.
本文提出了一种超声视频解释算法，该算法能够随着时间的推移添加新的类别或实例，而不会对先前表示的预测能力造成显著影响。其主要应用场景是诊断胎儿超声心动图分析。目前，在临床实践中，完整的诊断胎儿超声心动图记录并不常见。诊断视频的长度不一，且总结了一些不同难度的诊断子任务。虽然在开始时可能有大量的临床数据集可以用于构建基于超声图像的模型以进行自动图像分析，但随着时间的推移，数据也可能会变得可用，以协助算法的改进。为了解决这一问题，我们提出了一种增量学习方法，用于构建一个层次网络模型，该模型允许在无需先验数据分布的情况下并行地包含以前未见过的解剖类别。超类是通过粗分类和细分类获得的，以允许模型通过模块化架构在一系列类别中自我组织解剖结构。我们证明，这种方法可以适用于新的变量数据分布，而不会显著影响以前学习到的表示。考虑了两种添加新数据的极端情况：（1）随着时间的推移，新类别的数据量和分布与以前可用的类别相似，（2）在未来的时间内，不平衡的数据集将在少数情况下进行学习。在这两种情况下，都不假定先前类别的数据可用。使用新类别上的细分类的增量准确性验证学习过程的演变，并将其与从经验丰富的超声医生标注的临床数据集上进行端到端迁移学习微调的模型的结果进行比较。后续学习的模块化降低了旧任务的未来精度折旧率，从平衡增量的 6.75%降低到 1.10%。在未来增量中，使用不平衡数据分布时，折旧率从 6.95%降低到 1.89%，同时在新添加的细分类中保持有竞争力的分类精度，在所有阶段的参数操作都在相同数量级。
