Computer Assisted Medical Interventions, German Cancer Research Center, Im Neuenheimer Feld 581, 69210, Heidelberg, Germany.
Medical Image Computing, German Cancer Research Center, Im Neuenheimer Feld 581, 69210, Heidelberg, Germany.
Int J Comput Assist Radiol Surg. 2018 Jun;13(6):925-933. doi: 10.1007/s11548-018-1772-0. Epub 2018 Apr 27.
Surgical data science is a new research field that aims to observe all aspects of the patient treatment process in order to provide the right assistance at the right time. Due to the breakthrough successes of deep learning-based solutions for automatic image annotation, the availability of reference annotations for algorithm training is becoming a major bottleneck in the field. The purpose of this paper was to investigate the concept of self-supervised learning to address this issue.
Our approach is guided by the hypothesis that unlabeled video data can be used to learn a representation of the target domain that boosts the performance of state-of-the-art machine learning algorithms when used for pre-training. Core of the method is an auxiliary task based on raw endoscopic video data of the target domain that is used to initialize the convolutional neural network (CNN) for the target task. In this paper, we propose the re-colorization of medical images with a conditional generative adversarial network (cGAN)-based architecture as auxiliary task. A variant of the method involves a second pre-training step based on labeled data for the target task from a related domain. We validate both variants using medical instrument segmentation as target task.
The proposed approach can be used to radically reduce the manual annotation effort involved in training CNNs. Compared to the baseline approach of generating annotated data from scratch, our method decreases exploratively the number of labeled images by up to 75% without sacrificing performance. Our method also outperforms alternative methods for CNN pre-training, such as pre-training on publicly available non-medical (COCO) or medical data (MICCAI EndoVis2017 challenge) using the target task (in this instance: segmentation).
As it makes efficient use of available (non-)public and (un-)labeled data, the approach has the potential to become a valuable tool for CNN (pre-)training.
外科数据科学是一个新的研究领域，旨在观察患者治疗过程的各个方面，以便在正确的时间提供正确的帮助。由于基于深度学习的自动图像注释解决方案取得了突破，因此算法训练的参考注释可用性成为该领域的主要瓶颈。本文旨在研究自监督学习的概念来解决这个问题。
我们的方法受以下假设的指导：未标记的视频数据可用于学习目标域的表示，当用于预训练时，可提高最先进的机器学习算法的性能。该方法的核心是基于目标域的原始内窥镜视频数据的辅助任务，用于初始化目标任务的卷积神经网络（CNN）。在本文中，我们提出了使用基于条件生成对抗网络（cGAN）的架构对医学图像进行重新着色作为辅助任务。该方法的变体涉及基于来自相关领域的目标任务的标记数据进行的第二个预训练步骤。我们使用医学仪器分割作为目标任务来验证这两种变体。
所提出的方法可用于从根本上减少训练 CNN 所涉及的手动注释工作。与从头开始生成注释数据的基线方法相比，我们的方法在不牺牲性能的情况下，将标记图像的数量最多减少 75%。我们的方法还优于替代的 CNN 预训练方法，例如在目标任务（在这种情况下是分割）上使用目标任务（在这种情况下是分割）上使用公共的非医学（COCO）或医学数据（MICCAI EndoVis2017 挑战赛）进行预训练。
由于它有效地利用了可用的（非）公共和（非）标记数据，因此该方法有可能成为 CNN（预）训练的有价值工具。
