Dascena, Inc., Houston, TX, United States.
Dascena, Inc., Houston, TX, United States.
Thromb Res. 2022 Aug;216:14-21. doi: 10.1016/j.thromres.2022.05.016. Epub 2022 Jun 2.
Pulmonary embolism (PE) is a life-threatening condition associated with ~10% of deaths of hospitalized patients. Machine learning algorithms (MLAs) which predict the onset of pulmonary embolism (PE) could enable earlier treatment and improve patient outcomes. However, the extent to which they generalize to broader patient populations impacts their clinical utility.
To conduct the first large-scale external validation of a machine learning-based PE prediction model which uses EHR data from the first three hours of a patient's hospital stay to predict the occurrence of PE within the next 10 days of the inpatient stay.
This retrospective study included approximately two million adult hospital admissions across 44 medical institutions in the US from 2011 to 2017. Demographics, vital signs, and lab tests from adult inpatients at 12 institutions (n = 331,268; 3.3% PE positive) were used for training an XGBoost model. External validation of the model was conducted on patient populations from each of 32 medical institutions (total n = 1,660,715; 3.7% PE positive) without retraining. Model performance was assessed using the area under the receiver operating characteristic curve (AUROC). Backward elimination regression was used to identify correlations between characteristics of the external validation sets and AUROC.
The model performed well (AUROC = 0.87) on the 20% hold-out subset of the training set. Despite demographic differences between the 32 external validation populations (percent PE positive: min = 1.54%, max = 6.47%), without retraining, the model had excellent discrimination, with a mean AUROC of 0.88 (min = 0.79, max = 0.93). Fixing sensitivity at 0.80, the model had a mean specificity of 0.85 (min = 0.64, max = 0.93). Backward elimination regression identified a negative association (β = -0.015, p < 0.001) between the percentage of PE positive encounters and AUROC.
A PE prediction model performed remarkably well across 32 different external patient populations without retraining and despite significant differences in demographic characteristics, demonstrating its generalizability and potential as a clinical decision support tool to aid PE detection and improve patient outcomes in a clinical setting.
肺栓塞（PE）是一种危及生命的疾病，约有 10%的住院患者死亡与之相关。预测肺栓塞（PE）发生的机器学习算法（MLAs）可以实现早期治疗并改善患者预后。然而，它们在多大程度上能够推广到更广泛的患者群体，会影响其临床实用性。
对一个基于机器学习的 PE 预测模型进行首次大规模外部验证，该模型使用患者住院前 3 小时的电子健康记录（EHR）数据，预测住院后 10 天内发生 PE 的可能性。
本回顾性研究纳入了 2011 年至 2017 年期间美国 44 家医疗机构的约 200 万例成年住院患者。12 家医疗机构（n=331268；PE 阳性率为 3.3%）的成年住院患者的人口统计学、生命体征和实验室检查数据用于训练 XGBoost 模型。在不进行重新训练的情况下，对来自 32 家医疗机构（总 n=1660715；PE 阳性率为 3.7%）的患者人群进行模型外部验证。使用受试者工作特征曲线下面积（AUROC）评估模型性能。采用向后逐步消除回归法确定外部验证集特征与 AUROC 之间的相关性。
该模型在训练集的 20%留一验证子集上表现良好（AUROC=0.87）。尽管 32 个外部验证人群的人群特征存在差异（PE 阳性率：最小=1.54%，最大=6.47%），但无需重新训练，该模型的区分度仍极佳，平均 AUROC 为 0.88（最小=0.79，最大=0.93）。当固定敏感度为 0.80 时，该模型的平均特异性为 0.85（最小=0.64，最大=0.93）。向后逐步消除回归发现，PE 阳性病例百分比与 AUROC 呈负相关（β=-0.015，p<0.001）。
无需重新训练，该 PE 预测模型在 32 个不同的外部患者人群中表现出色，尽管在人口统计学特征方面存在显著差异，但仍具有很好的泛化能力，有望成为一种临床决策支持工具，用于辅助 PE 检测并改善临床环境中的患者预后。
