Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
Memorial Sloan Kettering Cancer Center, New York, NY, USA.
Med Phys. 2018 Oct;45(10):4568-4581. doi: 10.1002/mp.13141. Epub 2018 Sep 19.
This report presents the methods and results of the Thoracic Auto-Segmentation Challenge organized at the 2017 Annual Meeting of American Association of Physicists in Medicine. The purpose of the challenge was to provide a benchmark dataset and platform for evaluating performance of autosegmentation methods of organs at risk (OARs) in thoracic CT images.
Sixty thoracic CT scans provided by three different institutions were separated into 36 training, 12 offline testing, and 12 online testing scans. Eleven participants completed the offline challenge, and seven completed the online challenge. The OARs were left and right lungs, heart, esophagus, and spinal cord. Clinical contours used for treatment planning were quality checked and edited to adhere to the RTOG 1106 contouring guidelines. Algorithms were evaluated using the Dice coefficient, Hausdorff distance, and mean surface distance. A consolidated score was computed by normalizing the metrics against interrater variability and averaging over all patients and structures.
The interrater study revealed highest variability in Dice for the esophagus and spinal cord, and in surface distances for lungs and heart. Five out of seven algorithms that participated in the online challenge employed deep-learning methods. Although the top three participants using deep learning produced the best segmentation for all structures, there was no significant difference in the performance among them. The fourth place participant used a multi-atlas-based approach. The highest Dice scores were produced for lungs, with averages ranging from 0.95 to 0.98, while the lowest Dice scores were produced for esophagus, with a range of 0.55-0.72.
The results of the challenge showed that the lungs and heart can be segmented fairly accurately by various algorithms, while deep-learning methods performed better on the esophagus. Our dataset together with the manual contours for all training cases continues to be available publicly as an ongoing benchmarking resource.
本报告介绍了在 2017 年美国医学物理学家协会年会上组织的胸部分割挑战赛的方法和结果。该挑战赛的目的是提供一个基准数据集和平台，用于评估在胸部 CT 图像中对危及器官（OARs）进行自动分割方法的性能。
来自三个不同机构的 60 个胸部 CT 扫描被分为 36 个训练扫描、12 个离线测试扫描和 12 个在线测试扫描。11 名参与者完成了离线挑战，7 名参与者完成了在线挑战。OARs 包括左肺、右肺、心脏、食管和脊髓。用于治疗计划的临床轮廓经过质量检查和编辑，以符合 RTOG 1106 轮廓指南。使用 Dice 系数、Hausdorff 距离和平均表面距离来评估算法。通过将指标与组内变异性进行归一化，并对所有患者和结构进行平均，计算出综合得分。
组内研究显示，食管和脊髓的 Dice 差异最大，肺和心脏的表面距离差异最大。参加在线挑战的 7 种算法中有 5 种采用了深度学习方法。尽管使用深度学习的前三名参与者对所有结构的分割效果最好，但它们之间的性能没有显著差异。第四名参与者使用了一种基于多图谱的方法。肺的 Dice 得分最高，平均范围为 0.95 至 0.98，而食管的 Dice 得分最低，范围为 0.55 至 0.72。
挑战赛的结果表明，各种算法可以相当准确地分割肺和心脏，而深度学习方法在食管上的表现更好。我们的数据集以及所有训练案例的手动轮廓图仍然作为一个持续的基准资源公开提供。
