Radiology Informatics Laboratory, Department of Radiology, Mayo Clinic, Rochester, MN, USA.
Oslo University Hospital, Oslo, Norway.
J Digit Imaging. 2019 Aug;32(4):571-581. doi: 10.1007/s10278-019-00232-0.
Deep-learning algorithms typically fall within the domain of supervised artificial intelligence and are designed to "learn" from annotated data. Deep-learning models require large, diverse training datasets for optimal model convergence. The effort to curate these datasets is widely regarded as a barrier to the development of deep-learning systems. We developed RIL-Contour to accelerate medical image annotation for and with deep-learning. A major goal driving the development of the software was to create an environment which enables clinically oriented users to utilize deep-learning models to rapidly annotate medical imaging. RIL-Contour supports using fully automated deep-learning methods, semi-automated methods, and manual methods to annotate medical imaging with voxel and/or text annotations. To reduce annotation error, RIL-Contour promotes the standardization of image annotations across a dataset. RIL-Contour accelerates medical imaging annotation through the process of annotation by iterative deep learning (AID). The underlying concept of AID is to iteratively annotate, train, and utilize deep-learning models during the process of dataset annotation and model development. To enable this, RIL-Contour supports workflows in which multiple-image analysts annotate medical images, radiologists approve the annotations, and data scientists utilize these annotations to train deep-learning models. To automate the feedback loop between data scientists and image analysts, RIL-Contour provides mechanisms to enable data scientists to push deep newly trained deep-learning models to other users of the software. RIL-Contour and the AID methodology accelerate dataset annotation and model development by facilitating rapid collaboration between analysts, radiologists, and engineers.
深度学习算法通常属于监督人工智能领域，旨在“学习”带注释的数据。深度学习模型需要大型、多样化的训练数据集，以实现最佳模型收敛。精心制作这些数据集的工作被广泛认为是深度学习系统发展的障碍。我们开发了 RIL-Contour，以加速医学图像的深度学习标注。该软件的主要开发目标是创建一个环境，使面向临床的用户能够利用深度学习模型快速对医学成像进行标注。RIL-Contour 支持使用全自动深度学习方法、半自动方法和手动方法对医学成像进行体素和/或文本标注。为了减少标注错误，RIL-Contour 促进了数据集内图像标注的标准化。RIL-Contour 通过迭代深度学习（AID）的标注过程来加速医学图像标注。AID 的基本概念是在数据集标注和模型开发过程中迭代地进行标注、训练和利用深度学习模型。为了实现这一点，RIL-Contour 支持多个图像分析师标注医学图像、放射科医生批准标注以及数据科学家利用这些标注来训练深度学习模型的工作流程。为了在数据科学家和图像分析师之间自动反馈循环，RIL-Contour 提供了一些机制，使数据科学家能够将新训练的深度学习模型推送给软件的其他用户。RIL-Contour 和 AID 方法通过促进分析师、放射科医生和工程师之间的快速协作，加速了数据集标注和模型开发。
