Leone Alexandra O, Mohamed Abdallah S R, Fuller Clifton D, Peterson Christine B, Garden Adam S, Lee Anna, Mayo Lauren L, Moreno Amy C, Reddy Jay P, Hoffman Karen, Niedzielski Joshua S, Court Laurence E, Whitaker Thomas J
Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas.
Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas.
Adv Radiat Oncol. 2024 May 1;9(8):101533. doi: 10.1016/j.adro.2024.101533. eCollection 2024 Aug.
Our purpose was to develop a clinically intuitive and easily understandable scoring method using statistical metrics to visually determine the quality of a radiation treatment plan.
Data from 111 patients with head and neck cancer were used to establish a percentile-based scoring system for treatment plan quality evaluation on both a plan-by-plan and objective-by-objective basis. The percentile scores for each clinical objective and the overall treatment plan score were then visualized using a daisy plot. To validate our scoring method, 6 physicians were recruited to assess 60 plans, each using a scoring table consisting of a 5-point Likert scale (with scores ≥3 considered passing). Spearman correlation analysis was conducted to assess the association between increasing treatment plan percentile rank and physician rating, with Likert scores of 1 and 2 representing clinically unacceptable plans, scores of 3 and 4 representing plans needing minor edits, and a score of 5 representing clinically acceptable plans. Receiver operating characteristic curve analysis was used to assess the scoring system's ability to quantify plan quality.
Of the 60 plans scored by the physicians, 8 were deemed as clinically acceptable; these plans had an 89.0th ± 14.5 percentile value using our scoring system. The plans needing minor edits or deemed unacceptable had more variation, with scores falling in the 62.6nd ± 25.1 percentile and 35.6th ± 25.7 percentile, respectively. The estimated Spearman correlation coefficient between the physician score and treatment plan percentile was 0.53 ( < .001), indicating a moderate but statistically significant correlation. Receiver operating characteristic curve analysis demonstrated discernment between acceptable and unacceptable plan quality, with an area under the curve of 0.76.
Our scoring system correlates with physician ratings while providing intuitive visual feedback for identifying good treatment plan quality, thereby indicating its utility in the quality assurance process.
我们的目的是开发一种临床直观且易于理解的评分方法，使用统计指标直观地确定放射治疗计划的质量。
使用来自111例头颈癌患者的数据，在逐个计划和逐个目标的基础上建立基于百分位数的评分系统，用于治疗计划质量评估。然后使用雏菊图将每个临床目标的百分位数分数和总体治疗计划分数可视化。为了验证我们的评分方法，招募了6名医生评估60个计划，每位医生使用由5点李克特量表组成的评分表（分数≥3视为合格）。进行Spearman相关性分析，以评估治疗计划百分位数排名增加与医生评分之间的关联，李克特分数1和2代表临床不可接受的计划，分数3和4代表需要小幅编辑的计划，分数5代表临床可接受的计划。使用受试者工作特征曲线分析来评估评分系统量化计划质量的能力。
在医生评分的60个计划中，8个被视为临床可接受；使用我们的评分系统，这些计划的百分位数为89.0 ± 14.5。需要小幅编辑或被视为不可接受的计划差异更大，分数分别落在第62.6 ± 25.1百分位数和第35.6 ± 25.7百分位数。医生评分与治疗计划百分位数之间的估计Spearman相关系数为0.53（<0.001），表明存在中等但具有统计学意义的相关性。受试者工作特征曲线分析表明可接受和不可接受的计划质量之间有辨别力，曲线下面积为0.76。
我们的评分系统与医生评分相关，同时为识别良好的治疗计划质量提供直观的视觉反馈，从而表明其在质量保证过程中的效用。
