De Kerf Geert, Claessens Michaël, Raouassi Fadoua, Mercier Carole, Stas Daan, Ost Piet, Dirix Piet, Verellen Dirk
Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium.
Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Antwerp, Belgium.
Phys Imaging Radiat Oncol. 2023 Sep 23;28:100494. doi: 10.1016/j.phro.2023.100494. eCollection 2023 Oct.
Clinical Artificial Intelligence (AI) implementations lack ground-truth when applied on real-world data. This study investigated how combined geometrical and dose-volume metrics can be used as performance monitoring tools to detect clinically relevant candidates for model retraining.
Fifty patients were analyzed for both AI-segmentation and planning. For AI-segmentation, geometrical (Standard Surface Dice 3 mm and Local Surface Dice 3 mm) and dose-volume based parameters were calculated for two organs (bladder and anorectum) to compare AI output against the clinically corrected structure. A Local Surface Dice was introduced to detect geometrical changes in the vicinity of the target volumes, while an Absolute Dose Difference (ADD) evaluation increased focus on dose-volume related changes. AI-planning performance was evaluated using clinical goal analysis in combination with volume and target overlap metrics.
The Local Surface Dice reported equal or lower values compared to the Standard Surface Dice (anorectum: (0.93 ± 0.11) vs (0.98 ± 0.04); bladder: (0.97 ± 0.06) vs (0.98 ± 0.04)). The ADD metric showed a difference of (0.9 ± 0.8)Gy for the anorectum . The bladder reported a difference of (0.7 ± 1.5)Gy. Mandatory clinical goals were fulfilled in 90 % of the DLP plans.
Combining dose-volume and geometrical metrics allowed detection of clinically relevant changes, applied to both auto-segmentation and auto-planning output and the Local Surface Dice was more sensitive to local changes compared to the Standard Surface Dice. This monitoring is able to evaluate AI behavior in clinical practice and allows candidate selection for active learning.
临床人工智能（AI）应用于真实世界数据时缺乏真实对照。本研究调查了如何将几何和剂量体积指标结合用作性能监测工具，以检测模型再训练的临床相关候选对象。
对50例患者进行AI分割和计划分析。对于AI分割，计算了两个器官（膀胱和直肠）的几何参数（标准表面骰子系数3mm和局部表面骰子系数3mm）以及基于剂量体积的参数，以将AI输出与临床校正结构进行比较。引入局部表面骰子系数以检测靶体积附近的几何变化，而绝对剂量差异（ADD）评估则更关注与剂量体积相关的变化。使用临床目标分析结合体积和靶区重叠指标评估AI计划性能。
与标准表面骰子系数相比，局部表面骰子系数报告的值相等或更低（直肠：(0.93 ± 0.11) 对 (0.98 ± 0.04)；膀胱：(0.97 ± 0.06) 对 (0.98 ± 0.04)）。ADD指标显示直肠的差异为(0.9 ± 0.8)Gy。膀胱的差异为(0.7 ± 1.5)Gy。90%的剂量长度乘积（DLP）计划实现了强制性临床目标。
结合剂量体积和几何指标能够检测临床相关变化，适用于自动分割和自动计划输出，并且与标准表面骰子系数相比，局部表面骰子系数对局部变化更敏感。这种监测能够评估临床实践中的AI行为，并允许选择主动学习的候选对象。
