Department of Radiation Oncology, Montefiore Medical Center, Bronx, NY, 10467, USA.
Albert Einstein College of Medicine, Bronx, NY, 10461, USA.
Radiat Oncol. 2024 May 31;19(1):69. doi: 10.1186/s13014-024-02451-4.
Multiple artificial intelligence (AI)-based autocontouring solutions have become available, each promising high accuracy and time savings compared with manual contouring. Before implementing AI-driven autocontouring into clinical practice, three commercially available CT-based solutions were evaluated.
The following solutions were evaluated in this work: MIM-ProtégéAI+ (MIM), Radformation-AutoContour (RAD), and Siemens-DirectORGANS (SIE). Sixteen organs were identified that could be contoured by all solutions. For each organ, ten patients that had manually generated contours approved by the treating physician (AP) were identified, totaling forty-seven different patients. CT scans in the supine position were acquired using a Siemens-SOMATOMgo 64-slice helical scanner and used to generate autocontours. Physician scoring of contour accuracy was performed by at least three physicians using a five-point Likert scale. Dice similarity coefficient (DSC), Hausdorff distance (HD) and mean distance to agreement (MDA) were calculated comparing AI contours to "ground truth" AP contours.
The average physician score ranged from 1.00, indicating that all physicians reviewed the contour as clinically acceptable with no modifications necessary, to 3.70, indicating changes are required and that the time taken to modify the structures would likely take as long or longer than manually generating the contour. When averaged across all sixteen structures, the AP contours had a physician score of 2.02, MIM 2.07, RAD 1.96 and SIE 1.99. DSC ranged from 0.37 to 0.98, with 41/48 (85.4%) contours having an average DSC ≥ 0.7. Average HD ranged from 2.9 to 43.3 mm. Average MDA ranged from 0.6 to 26.1 mm.
The results of our comparison demonstrate that each vendor's AI contouring solution exhibited capabilities similar to those of manual contouring. There were a small number of cases where unusual anatomy led to poor scores with one or more of the solutions. The consistency and comparable performance of all three vendors' solutions suggest that radiation oncology centers can confidently choose any of the evaluated solutions based on individual preferences, resource availability, and compatibility with their existing clinical workflows. Although AI-based contouring may result in high-quality contours for the majority of patients, a minority of patients require manual contouring and more in-depth physician review.
多种基于人工智能（AI）的自动勾画解决方案已经问世，与手动勾画相比，每种方案都承诺具有更高的准确性和时间效率。在将 AI 驱动的自动勾画应用于临床实践之前，我们对三种市售 CT 基础解决方案进行了评估。
本研究评估了以下三种解决方案：MIM-ProtégéAI+（MIM）、Radformation-AutoContour（RAD）和 Siemens-DirectORGANS（SIE）。可以对所有解决方案进行勾画的 16 种器官被识别出来。对于每种器官，我们都找到了 10 位手动勾画且得到治疗医生认可的患者（AP），总计涉及 47 位不同的患者。仰卧位 CT 扫描使用西门子 SOMATOMgo 64 排螺旋扫描仪采集，用于生成自动勾画。至少有三位医生使用 5 分李克特量表对勾画的准确性进行评分。通过计算 Dice 相似系数（DSC）、Hausdorff 距离（HD）和平均吻合距离（MDA），将 AI 勾画与“金标准”AP 勾画进行比较。
平均医生评分范围从 1.00（表示所有医生认为勾画可直接用于临床，无需任何修改）到 3.70（表示需要修改，且修改结构所需的时间可能与手动勾画所需的时间一样长或更长）。当平均所有 16 种结构时，AP 勾画的医生评分为 2.02，MIM 为 2.07，RAD 为 1.96，SIE 为 1.99。DSC 范围为 0.37 至 0.98，48 个勾画中有 41 个（85.4%）的平均 DSC≥0.7。平均 HD 范围为 2.9 至 43.3mm。平均 MDA 范围为 0.6 至 26.1mm。
我们的比较结果表明，每个供应商的 AI 勾画解决方案都表现出与手动勾画相似的能力。在少数情况下，由于解剖结构异常，一个或多个解决方案的评分较低。所有三个供应商的解决方案的一致性和可比性能表明，放射肿瘤学中心可以根据个人偏好、资源可用性以及与现有临床工作流程的兼容性，有信心地选择评估的任何解决方案。虽然基于 AI 的勾画可能会为大多数患者生成高质量的勾画，但少数患者需要手动勾画和更深入的医生审查。
