Department of Radiation Medicine, University of Kentucky, Lexington, Kentucky, USA.
Carina Medical LLC, Lexington, Kentucky, USA.
Med Phys. 2023 May;50(5):2715-2732. doi: 10.1002/mp.16299. Epub 2023 Feb 25.
Contouring error is one of the top failure modes in radiation treatment. Multiple efforts have been made to develop tools to automatically detect segmentation errors. Deep learning-based auto-segmentation (DLAS) has been used as a baseline for flagging manual segmentation errors, but those efforts are limited to using only one or two contour comparison metrics.
The purpose of this research is to develop an improved contouring quality assurance system to identify and flag manual contouring errors.
DLAS contours were used as a reference to compare with manually segmented contours. A total of 27 geometric agreement metrics were determined from the comparisons between the two segmentation approaches. Feature selection was performed to optimize the training of a machine learning classification model to identify potential contouring errors. A public dataset with 339 cases was used to train and test the classifier. Four independent classifiers were trained using five-fold cross validation, and the predictions from each classifier were ensembled using soft voting. The trained model was validated on a held-out testing dataset. An additional independent clinical dataset with 60 cases was used to test the generalizability of the model. Model predictions were reviewed by an expert to confirm or reject the findings.
The proposed machine learning multiple features (ML-MF) approach outperformed traditional nonmachine-learning-based approaches that are based on only one or two geometric agreement metrics. The machine learning model achieved recall (precision) values of 0.842 (0.899), 0.762 (0.762), 0.727 (0.842), and 0.773 (0.773) for Brainstem, Parotid_L, Parotid_R, and mandible contours, respectively compared to 0.526 (0.909), 0.619 (0.765), 0.682 (0.882), 0.773 (0.568) for an approach based solely on Dice similarity coefficient values. In the external validation dataset, 66.7, 93.3, 94.1, and 58.8% of flagged cases were confirmed to have contouring errors by an expert for Brainstem, Parotid_L, Parotid_R, and mandible contours, respectively.
The proposed ML-MF approach, which includes multiple geometric agreement metrics to flag manual contouring errors, demonstrated superior performance in comparison to traditional methods. This method is easy to implement in clinical practice and can help to reduce the significant time and labor costs associated with manual segmentation and review.
轮廓误差是放射治疗中最主要的失效模式之一。已经做了多种努力来开发工具以自动检测分割错误。基于深度学习的自动分割（DLAS）已被用作标记手动分割错误的基准，但这些努力仅限于使用一个或两个轮廓比较指标。
本研究旨在开发一种改进的轮廓质量保证系统，以识别和标记手动轮廓错误。
DLAS 轮廓被用作参考，与手动分割轮廓进行比较。从两种分割方法的比较中确定了总共 27 个几何一致性度量。进行了特征选择以优化机器学习分类模型的训练，以识别潜在的轮廓错误。使用 339 例公共数据集对分类器进行训练和测试。使用五折交叉验证训练了四个独立的分类器，并使用软投票对每个分类器的预测进行集成。在保留的测试数据集上验证了训练好的模型。使用另外一个 60 例的独立临床数据集来测试模型的泛化能力。由专家审查模型预测以确认或拒绝发现。
所提出的基于机器学习的多特征（ML-MF）方法优于仅基于一个或两个几何一致性度量的传统非基于机器学习的方法。与基于 Dice 相似系数值的方法（0.526（0.909），0.619（0.765），0.682（0.882），0.773（0.568）相比，机器学习模型在脑干、腮腺 L、腮腺 R 和下颌骨轮廓上的召回率（精度）值分别为 0.842（0.899），0.762（0.762），0.727（0.842）和 0.773（0.773）。在外部验证数据集中，专家确认脑干、腮腺 L、腮腺 R 和下颌骨轮廓的标记病例中分别有 66.7%、93.3%、94.1%和 58.8%存在轮廓错误。
所提出的 ML-MF 方法包括多个几何一致性度量，用于标记手动轮廓错误，与传统方法相比表现出更好的性能。这种方法易于在临床实践中实施，可以帮助减少与手动分割和审查相关的大量时间和劳动力成本。
