Department of Emergency Medicine, Seoul National University Hospital, Seoul, Korea.
Interdisciplinary Program in Bioengineering, Graduate School, Seoul National University, Seoul, Korea.
J Korean Med Sci. 2021 Jul 19;36(28):e187. doi: 10.3346/jkms.2021.36.e187.
BACKGROUND: We performed this study to establish a prediction model for 1-year neurological outcomes in out-of-hospital cardiac arrest (OHCA) patients who achieved return of spontaneous circulation (ROSC) immediately after ROSC using machine learning methods. METHODS: We performed a retrospective analysis of an OHCA survivor registry. Patients aged ≥ 18 years were included. Study participants who had registered between March 31, 2013 and December 31, 2018 were divided into a develop dataset (80% of total) and an internal validation dataset (20% of total), and those who had registered between January 1, 2019 and December 31, 2019 were assigned to an external validation dataset. Four machine learning methods, including random forest, support vector machine, ElasticNet and extreme gradient boost, were implemented to establish prediction models with the develop dataset, and the ensemble technique was used to build the final prediction model. The prediction performance of the model in the internal validation and the external validation dataset was described with accuracy, area under the receiver-operating characteristic curve, area under the precision-recall curve, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Futhermore, we established multivariable logistic regression models with the develop set and compared prediction performance with the ensemble models. The primary outcome was an unfavorable 1-year neurological outcome. RESULTS: A total of 1,207 patients were included in the study. Among them, 631, 139, and 153 were assigned to the develop, the internal validation and the external validation datasets, respectively. Prediction performance metrics for the ensemble prediction model in the internal validation dataset were as follows: accuracy, 0.9620 (95% confidence interval [CI], 0.9352-0.9889); area under receiver-operator characteristics curve, 0.9800 (95% CI, 0.9612-0.9988); area under precision-recall curve, 0.9950 (95% CI, 0.9860-1.0000); sensitivity, 0.9594 (95% CI, 0.9245-0.9943); specificity, 0.9714 (95% CI, 0.9162-1.0000); PPV, 0.9916 (95% CI, 0.9752-1.0000); NPV, 0.8718 (95% CI, 0.7669-0.9767). Prediction performance metrics for the model in the external validation dataset were as follows: accuracy, 0.8509 (95% CI, 0.7825-0.9192); area under receiver-operator characteristics curve, 0.9301 (95% CI, 0.8845-0.9756); area under precision-recall curve, 0.9476 (95% CI, 0.9087-0.9867); sensitivity, 0.9595 (95% CI, 0.9145-1.0000); specificity, 0.6500 (95% CI, 0.5022-0.7978); PPV, 0.8353 (95% CI, 0.7564-0.9142); NPV, 0.8966 (95% CI, 0.7857-1.0000). All the prediction metrics were higher in the ensemble models, except NPVs in both the internal and the external validation datasets. CONCLUSION: We established an ensemble prediction model for prediction of unfavorable 1-year neurological outcomes in OHCA survivors using four machine learning methods. The prediction performance of the ensemble model was higher than the multivariable logistic regression model, while its performance was slightly decreased in the external validation dataset.
背景:我们使用机器学习方法,对即刻实现自主循环恢复(ROSC)的院外心脏骤停(OHCA)患者进行了 1 年神经功能预后的预测模型的建立。
方法:我们对 OHCA 幸存者登记处进行了回顾性分析。纳入年龄≥18 岁的患者。将 2013 年 3 月 31 日至 2018 年 12 月 31 日期间登记的患者分为开发数据集(总样本的 80%)和内部验证数据集(总样本的 20%),2019 年 1 月 1 日至 2019 年 12 月 31 日期间登记的患者被分配到外部验证数据集。我们使用四种机器学习方法,包括随机森林、支持向量机、ElasticNet 和极端梯度提升,来建立开发数据集的预测模型,并使用集成技术构建最终的预测模型。模型在内部验证和外部验证数据集的预测性能用准确率、接受者操作特征曲线下面积、精确召回曲线下面积、敏感度、特异性、阳性预测值(PPV)和阴性预测值(NPV)来描述。此外,我们使用开发集建立了多变量逻辑回归模型,并与集成模型的预测性能进行了比较。主要结局为 1 年不良神经结局。
结果:共有 1207 例患者纳入研究。其中,631 例、139 例和 153 例分别被分配到开发、内部验证和外部验证数据集。在内部验证数据集中,集成预测模型的预测性能指标如下:准确率为 0.9620(95%置信区间[CI],0.9352-0.9889);接受者操作特征曲线下面积为 0.9800(95%CI,0.9612-0.9988);精确召回曲线下面积为 0.9950(95%CI,0.9860-1.0000);敏感度为 0.9594(95%CI,0.9245-0.9943);特异性为 0.9714(95%CI,0.9162-1.0000);PPV 为 0.9916(95%CI,0.9752-1.0000);NPV 为 0.8718(95%CI,0.7669-0.9767)。在外部验证数据集中,模型的预测性能指标如下:准确率为 0.8509(95%CI,0.7825-0.9192);接受者操作特征曲线下面积为 0.9301(95%CI,0.8845-0.9756);精确召回曲线下面积为 0.9476(95%CI,0.9087-0.9867);敏感度为 0.9595(95%CI,0.9145-1.0000);特异性为 0.6500(95%CI,0.5022-0.7978);PPV 为 0.8353(95%CI,0.7564-0.9142);NPV 为 0.8966(95%CI,0.7857-1.0000)。除了内部和外部验证数据集的 NPV 外,所有预测指标在集成模型中均较高。
结论:我们使用四种机器学习方法建立了一个用于预测 OHCA 幸存者 1 年不良神经结局的集成预测模型。集成模型的预测性能高于多变量逻辑回归模型,而在外部验证数据集的预测性能略有下降。
Resusc Plus. 2025-5-3
J Med Internet Res. 2025-3-10
Circ Cardiovasc Qual Outcomes. 2019-7-9