Department of Anesthesiology and Pain Medicine, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul, 03080, Republic of Korea.
VUNO, 479 Gangnam-Daero, Seocho-gu, Seoul, 06541, Republic of Korea.
Crit Care. 2024 Mar 14;28(1):76. doi: 10.1186/s13054-024-04866-7.
A real-time model for predicting short-term mortality in critically ill patients is needed to identify patients at imminent risk. However, the performance of the model needs to be validated in various clinical settings and ethnicities before its clinical application. In this study, we aim to develop an ensemble machine learning model using routinely measured clinical variables at a single academic institution in South Korea.
We developed an ensemble model using deep learning and light gradient boosting machine models. Internal validation was performed using the last two years of the internal cohort dataset, collected from a single academic hospital in South Korea between 2007 and 2021. External validation was performed using the full Medical Information Mart for Intensive Care (MIMIC), eICU-Collaborative Research Database (eICU-CRD), and Amsterdam University Medical Center database (AmsterdamUMCdb) data. The area under the receiver operating characteristic curve (AUROC) was calculated and compared to that for the National Early Warning Score (NEWS).
The developed model (iMORS) demonstrated high predictive performance with an internal AUROC of 0.964 (95% confidence interval [CI] 0.963-0.965) and external AUROCs of 0.890 (95% CI 0.889-0.891) for MIMIC, 0.886 (95% CI 0.885-0.887) for eICU-CRD, and 0.870 (95% CI 0.868-0.873) for AmsterdamUMCdb. The model outperformed the NEWS with higher AUROCs in the internal and external validation (0.866 for the internal, 0.746 for MIMIC, 0.798 for eICU-CRD, and 0.819 for AmsterdamUMCdb; p < 0.001).
Our real-time machine learning model to predict short-term mortality in critically ill patients showed excellent performance in both internal and external validations. This model could be a useful decision-support tool in the intensive care units to assist clinicians.
需要建立一个实时模型来预测危重症患者的短期死亡率，以便识别即将面临风险的患者。然而，在将该模型应用于临床之前，需要在不同的临床环境和种族中对其性能进行验证。本研究旨在使用韩国某一学术机构的常规测量临床变量来开发一个集成机器学习模型。
我们使用深度学习和轻梯度提升机模型来开发集成模型。使用内部队列数据集的最后两年进行内部验证，该数据集是在 2007 年至 2021 年间从韩国的一家学术医院收集的。使用全医疗信息监护病房（MIMIC）、重症监护电子病历协作研究数据库（eICU-CRD）和阿姆斯特丹大学医学中心数据库（AmsterdamUMCdb）进行外部验证。计算了接受者操作特征曲线下的面积（AUROC）并与国家早期预警评分（NEWS）进行了比较。
开发的模型（iMORS）表现出较高的预测性能，内部 AUROC 为 0.964（95%置信区间 0.963-0.965），外部 MIMIC 的 AUROC 为 0.890（95%置信区间 0.889-0.891），eICU-CRD 的 AUROC 为 0.886（95%置信区间 0.885-0.887），AmsterdamUMCdb 的 AUROC 为 0.870（95%置信区间 0.868-0.873）。该模型在内部和外部验证中均优于 NEWS，AUROC 更高（内部为 0.866，MIMIC 为 0.746，eICU-CRD 为 0.798，AmsterdamUMCdb 为 0.819；p<0.001）。
我们开发的用于预测危重症患者短期死亡率的实时机器学习模型在内部和外部验证中表现出优异的性能。该模型可为重症监护病房的临床医生提供一种有用的决策支持工具。
