Poel Robert, Rüfenacht Elias, Scheib Stefan, Hemmatazad Hossein, Krcek Reinhardt, Tran Sébastien, Romano Edourd, Rogers Susanne, Stieb Sonja, Poolakundan Mohamed Riyas, Al-Abdulla Hissa Hussein, Foerster Robert, Schröder Christina, Oehler Christoph, Hong Julian, Breedveld Sebastiaan, Andratschke Nicolaus, Manser Peter, Fix Michael K, Aebersold Daniel M, Reyes Mauricio, Ermiş Ekin
Department of Radiation Oncology, Inselspital, Bern University Hospital, and University of Bern, Bern, Switzerland.
ARTORG Center for Biomedical Research, University of Bern, Bern, Switzerland.
Commun Med (Lond). 2025 Jul 31;5(1):319. doi: 10.1038/s43856-025-01048-6.
Manual contouring of organs at risk in radiotherapy is time-consuming, taking 1-4 hours per case. Automatic segmentation using deep learning has emerged as a promising solution, with many commercial options now available. However, these methods require rigorous validation before clinical use, and current evaluation approaches lack consistency and comprehensive assessment across publications.
We developed the Comprehensive Multifaceted Technical Evaluation framework, which integrates four key assessment components: quantitative geometric measures, qualitative expert evaluation, time efficiency analysis, and dosimetric evaluation. We demonstrated this framework using an in-house automatic segmentation model for brain organs at risk, trained on 100 cases and evaluated by 8 radiation oncology experts from 4 institutions. The evaluation included geometric accuracy measurements, expert ratings of clinical acceptability, time-saving assessments, and dosimetric impact analysis comparing treatment plans.
Here we show that our automatic segmentation model achieved an overall geometric accuracy of 0.78 and outperformed manual inter-rater variability. Expert evaluation revealed that 88% of automatically segmented structures were clinically acceptable with only minor adjustments needed. The evaluation and adjustment process averaged 22 minutes compared to 69 minutes for manual contouring. Dosimetric analysis showed minimal impact on treatment plans, with average dose differences of 0.30 Gray for mean dose and 0.23 Gray for maximum dose.
The framework provides a robust method for validating automatic segmentation models in radiotherapy. However, establishing standardized benchmarks and consensus guidelines within the radiotherapy community remains essential for proper clinical implementation and comparison of different segmentation tools.
在放射治疗中，手动勾勒危及器官的轮廓非常耗时，每个病例需要1至4小时。使用深度学习的自动分割技术已成为一种有前景的解决方案，现在有许多商业选项可供选择。然而，这些方法在临床应用前需要进行严格验证，并且目前的评估方法在不同出版物之间缺乏一致性和全面性。
我们开发了综合多方面技术评估框架，该框架整合了四个关键评估要素：定量几何测量、定性专家评估、时间效率分析和剂量学评估。我们使用一个内部自动分割模型对脑危及器官进行了演示，该模型基于100个病例进行训练，并由来自4个机构的8名放射肿瘤学专家进行评估。评估包括几何准确性测量、临床可接受性的专家评分、节省时间评估以及比较治疗计划的剂量学影响分析。
我们在此表明，我们的自动分割模型总体几何准确性达到0.78，优于手动评分者间的变异性。专家评估显示，88%的自动分割结构在临床上是可接受的，只需进行少量调整。评估和调整过程平均耗时22分钟，而手动勾勒轮廓则需要69分钟。剂量学分析表明对治疗计划的影响最小，平均剂量差异为平均剂量0.30戈瑞，最大剂量0.23戈瑞。
该框架为验证放射治疗中的自动分割模型提供了一种可靠的方法。然而，在放射治疗界建立标准化基准和共识指南对于不同分割工具的正确临床实施和比较仍然至关重要。
