Department of Anesthesiology and Critical Care, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, United States of America.
Penn Center for Perioperative Outcomes Research and Transformation (CPORT), University of Pennsylvania, Philadelphia, Pennsylvania, United States of America.
PLoS One. 2021 Jun 3;16(6):e0252585. doi: 10.1371/journal.pone.0252585. eCollection 2021.
This study aimed to develop and validate a claims-based, machine learning algorithm to predict clinical outcomes across both medical and surgical patient populations.
This retrospective, observational cohort study, used a random 5% sample of 770,777 fee-for-service Medicare beneficiaries with an inpatient hospitalization between 2009-2011. The machine learning algorithms tested included: support vector machine, random forest, multilayer perceptron, extreme gradient boosted tree, and logistic regression. The extreme gradient boosted tree algorithm outperformed the alternatives and was the machine learning method used for the final risk model. Primary outcome was 30-day mortality. Secondary outcomes were: rehospitalization, and any of 23 adverse clinical events occurring within 30 days of the index admission date.
The machine learning algorithm performance was evaluated by both the area under the receiver operating curve (AUROC) and Brier Score. The risk model demonstrated high performance for prediction of: 30-day mortality (AUROC = 0.88; Brier Score = 0.06), and 17 of the 23 adverse events (AUROC range: 0.80-0.86; Brier Score range: 0.01-0.05). The risk model demonstrated moderate performance for prediction of: rehospitalization within 30 days (AUROC = 0.73; Brier Score: = 0.07) and six of the 23 adverse events (AUROC range: 0.74-0.79; Brier Score range: 0.01-0.02). The machine learning risk model performed comparably on a second, independent validation dataset, confirming that the risk model was not overfit.
We have developed and validated a robust, claims-based, machine learning risk model that is applicable to both medical and surgical patient populations and demonstrates comparable predictive accuracy to existing risk models.
本研究旨在开发和验证一种基于索赔的机器学习算法,以预测医疗和手术患者群体的临床结局。
本回顾性观察队列研究使用了 2009-2011 年间住院的 770777 名付费医疗保险受益人的随机 5%样本。测试的机器学习算法包括:支持向量机、随机森林、多层感知机、极端梯度提升树和逻辑回归。极端梯度提升树算法表现优于其他算法,是最终风险模型使用的机器学习方法。主要结局为 30 天死亡率。次要结局为:30 天内再次住院和 23 种不良临床事件中的任何一种在索引入院日期后 30 天内发生。
通过接收者操作特征曲线(AUROC)和 Brier 评分评估机器学习算法的性能。风险模型在预测 30 天死亡率(AUROC = 0.88;Brier 评分 = 0.06)和 23 种不良事件中的 17 种(AUROC 范围:0.80-0.86;Brier 评分范围:0.01-0.05)方面表现出较高的性能。风险模型在预测 30 天内再次住院(AUROC = 0.73;Brier 评分 = 0.07)和 23 种不良事件中的 6 种(AUROC 范围:0.74-0.79;Brier 评分范围:0.01-0.02)方面表现出中等性能。机器学习风险模型在第二个独立验证数据集上的表现相当,证实风险模型没有过度拟合。
我们已经开发并验证了一种强大的、基于索赔的机器学习风险模型,该模型适用于医疗和手术患者群体,并且与现有风险模型具有可比的预测准确性。