多类别卵巢恶性肿瘤风险模型：算法选择导致预测不确定性的说明。

Multiclass risk models for ovarian malignancy: an illustration of prediction uncertainty due to the choice of algorithm.

机构信息

Department of Development and Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000, Belgium.

Department of Oncology, Leuven Cancer Institute, Laboratory of Tumor Immunology and Immunotherapy, KU Leuven, Leuven, Belgium.

出版信息

BMC Med Res Methodol. 2023 Nov 24;23(1):276. doi: 10.1186/s12874-023-02103-3.

DOI:10.1186/s12874-023-02103-3

PMID:38001421

原文链接:https://pmc.ncbi.nlm.nih.gov/articles/PMC10668424/

Abstract

BACKGROUND

Assessing malignancy risk is important to choose appropriate management of ovarian tumors. We compared six algorithms to estimate the probabilities that an ovarian tumor is benign, borderline malignant, stage I primary invasive, stage II-IV primary invasive, or secondary metastatic.

METHODS

This retrospective cohort study used 5909 patients recruited from 1999 to 2012 for model development, and 3199 patients recruited from 2012 to 2015 for model validation. Patients were recruited at oncology referral or general centers and underwent an ultrasound examination and surgery ≤ 120 days later. We developed models using standard multinomial logistic regression (MLR), Ridge MLR, random forest (RF), XGBoost, neural networks (NN), and support vector machines (SVM). We used nine clinical and ultrasound predictors but developed models with or without CA125.

RESULTS

Most tumors were benign (3980 in development and 1688 in validation data), secondary metastatic tumors were least common (246 and 172). The c-statistic (AUROC) to discriminate benign from any type of malignant tumor ranged from 0.89 to 0.92 for models with CA125, from 0.89 to 0.91 for models without. The multiclass c-statistic ranged from 0.41 (SVM) to 0.55 (XGBoost) for models with CA125, and from 0.42 (SVM) to 0.51 (standard MLR) for models without. Multiclass calibration was best for RF and XGBoost. Estimated probabilities for a benign tumor in the same patient often differed by more than 0.2 (20% points) depending on the model. Net Benefit for diagnosing malignancy was similar for algorithms at the commonly used 10% risk threshold, but was slightly higher for RF at higher thresholds. Comparing models, between 3% (XGBoost vs. NN, with CA125) and 30% (NN vs. SVM, without CA125) of patients fell on opposite sides of the 10% threshold.

CONCLUSION

Although several models had similarly good performance, individual probability estimates varied substantially.

摘要

背景

评估恶性肿瘤风险对于选择适当的卵巢肿瘤管理至关重要。我们比较了六种算法，以估计卵巢肿瘤为良性、交界性恶性、I 期原发性浸润性、II-IV 期原发性浸润性或继发性转移性的概率。

方法

本回顾性队列研究使用了 1999 年至 2012 年招募的 5909 名患者进行模型开发，以及 2012 年至 2015 年招募的 3199 名患者进行模型验证。患者在肿瘤转诊或普通中心招募，并在超声检查后≤120 天内进行手术。我们使用标准多项逻辑回归（MLR）、岭 MLR、随机森林（RF）、XGBoost、神经网络（NN）和支持向量机（SVM）开发模型。我们使用了九个临床和超声预测因子，但开发了包含或不包含 CA125 的模型。

结果

大多数肿瘤为良性（在开发数据和验证数据中分别为 3980 例和 1688 例），继发性转移性肿瘤最为罕见（分别为 246 例和 172 例）。有 CA125 的模型中，良性与任何类型恶性肿瘤的判别 c 统计量（AUROC）范围为 0.89 至 0.92，无 CA125 的模型为 0.89 至 0.91。有 CA125 的模型中，多类别 c 统计量范围为 0.41（SVM）至 0.55（XGBoost），无 CA125 的模型为 0.42（SVM）至 0.51（标准 MLR）。RF 和 XGBoost 对多类别校准效果最好。在同一患者中，由于模型不同，良性肿瘤的估计概率往往相差 0.2 以上（20%）。在常用的 10%风险阈值下，诊断恶性肿瘤的算法的净效益相似，但在较高阈值下，RF 的净效益略高。在模型比较中，有 3%（XGBoost 与 NN，有 CA125）和 30%（NN 与 SVM，无 CA125）的患者在 10%阈值的两侧分布情况截然不同。