Henderson Edward G A, Vasquez Osorio Eliana M, van Herk Marcel, Brouwer Charlotte L, Steenbakkers Roel J H M, Green Andrew F
Division of Cancer Sciences, The University of Manchester, M13 9PL Manchester, United Kingdom.
Department of Radiotherapy Related Research, The Christie NHS Foundation Trust, M20 4BX Manchester, United Kingdom.
Phys Med Biol. 2023 Apr 3;68(8). doi: 10.1088/1361-6560/acc309.
Automatic segmentation of organs-at-risk in radiotherapy planning computed tomography (CT) scans using convolutional neural networks (CNNs) is an active research area. Very large datasets are usually required to train such CNN models. In radiotherapy, large, high-quality datasets are scarce and combining data from several sources can reduce the consistency of training segmentations. It is therefore important to understand the impact of training data quality on the performance of auto-segmentation models for radiotherapy.In this study, we took an existing 3D CNN architecture for head and neck CT auto-segmentation and compare the performance of models trained with a small, well-curated dataset (= 34) and then a far larger dataset (= 185) containing less consistent training segmentations. We performed 5-fold cross-validations in each dataset and tested segmentation performance using the 95th percentile Hausdorff distance and mean distance-to-agreement metrics. Finally, we validated the generalisability of our models with an external cohort of patient data (= 12) with five expert annotators.The models trained with a large dataset were greatly outperformed by models (of identical architecture) trained with a smaller, but higher consistency set of training samples. Our models trained with a small dataset produce segmentations of similar accuracy as expert human observers and generalised well to new data, performing within inter-observer variation.We empirically demonstrate the importance of highly consistent training samples when training a 3D auto-segmentation model for use in radiotherapy. Crucially, it is the consistency of the training segmentations which had a greater impact on model performance rather than the size of the dataset used.
使用卷积神经网络（CNN）对放射治疗计划计算机断层扫描（CT）图像中的危及器官进行自动分割是一个活跃的研究领域。训练此类CNN模型通常需要非常大的数据集。在放射治疗中，大型高质量数据集稀缺，并且合并来自多个来源的数据会降低训练分割的一致性。因此，了解训练数据质量对放射治疗自动分割模型性能的影响非常重要。在本研究中，我们采用了一种现有的用于头颈部CT自动分割的3D CNN架构，并比较了使用小的、精心整理的数据集（=34）然后使用包含一致性较低的训练分割的大得多的数据集（=185）训练的模型的性能。我们在每个数据集中进行了5折交叉验证，并使用第95百分位数豪斯多夫距离和平均一致距离指标测试分割性能。最后，我们使用由五名专家注释者标注的外部患者数据队列（=12）验证了我们模型的泛化能力。使用大数据集训练的模型大大不如使用较小但一致性更高的训练样本集训练的（相同架构）模型。我们使用小数据集训练的模型产生的分割精度与专家人工观察者相似，并且对新数据具有良好的泛化能力，在观察者间差异范围内表现良好。我们通过实验证明了在训练用于放射治疗的3D自动分割模型时高度一致的训练样本的重要性。至关重要的是，对模型性能影响更大的是训练分割的一致性，而不是所使用数据集的大小。
