Department of Medical Informatics, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran.
Pharmaceutical Research Center, Mashhad University of Medical Sciences, Mashhad, Iran.
Biomed Res Int. 2022 Apr 25;2022:3964063. doi: 10.1155/2022/3964063. eCollection 2022.
A variety of scoring systems have been introduced for use in both the emergency department (ED) such as WPS, REMS, and MEWS and the intensive care unit (ICU) such as APACHE II, SAPS II, and SOFA for risk stratification and mortality prediction. However, the performance of these models in the ICU remains unclear and we aimed to evaluate and compare their performance in the ICU.
This multicenter retrospective cohort study was conducted on severely ill patients admitted to the ICU directly from the ED in seven tertiary hospitals in Iran from August 2018 to August 2020. We evaluated all models in terms of discrimination (AUROC), the balance between positive predictive value and sensitivity (AUPRC), calibration (Hosmer-Lemeshow test and calibration plots), and overall performance using the Brier score (BS). The endpoint was considered inhospital mortality.
Among the 3,455 patients included in the study, 54.4% of individuals were male ( = 1,879) and 26.5% deceased ( = 916). The BS for the WPS, REMS, MEWS, APACHE II, SAPS II, and SOFA were 0.178, 0.165, 0.183, 0.157, 0.170, and 0.182, respectively. The AUROC of these models were 0.728 (0.71-0.75), 0.761 (0.74-0.78), 0.682 (0.66-0.70), 0.810 (0.79-0.83), 0.767 (0.75-0.79), and 0.785 (0.77-0.80), respectively. The AUPRC was 0.517 (0.50-0.53) for WPS, 0.547 (0.53-0.56) for REMS, 0.445 (0.42-0.46) for MEWS, 0.630 (0.61-0.65) for APACHE II, 0.559 (0.54-0.58) for SAPS II, and 0.564 (0.54-0.57) for SOFA. All models except the MEWS and SOFA had good calibration. The most accurate model belonged to APACHE II with lowest BS.
The APACHE II outperformed all the ED and ICU models and was found to be the most appropriate model in predicting inhospital mortality of patients in the ICU in terms of discrimination, calibration, and accuracy of predicted probability. Except for MEWS, the rest of the models had fair discrimination and partially good calibration. Interestingly, although the REMS is less complicated than the SAPS II, both models exhibited similar performance. Clinicians can utilize the REMS as part of a larger clinical assessment to manage patients more effectively.
各种评分系统已被引入到急诊科（ED）和重症监护病房（ICU）中，用于风险分层和死亡率预测。在 ED 中使用的评分系统有 WPS、REMS 和 MEWS，在 ICU 中使用的评分系统有 APACHE II、SAPS II 和 SOFA 等。然而，这些模型在 ICU 中的表现尚不清楚，我们旨在评估和比较它们在 ICU 中的性能。
这是一项多中心回顾性队列研究，在 2018 年 8 月至 2020 年 8 月期间，在伊朗的 7 家三级医院中，对直接从 ED 转入 ICU 的重症患者进行了研究。我们从区分度（AUROC）、阳性预测值和敏感度之间的平衡（AUPRC）、校准（Hosmer-Lemeshow 检验和校准图）和整体表现（Brier 得分（BS））等方面对所有模型进行了评估。终点是院内死亡率。
在纳入的 3455 名患者中，54.4%（n=1879）为男性，26.5%（n=916）死亡。WPS、REMS、MEWS、APACHE II、SAPS II 和 SOFA 的 BS 分别为 0.178、0.165、0.183、0.157、0.170 和 0.182。这些模型的 AUROC 分别为 0.728（0.71-0.75）、0.761（0.74-0.78）、0.682（0.66-0.70）、0.810（0.79-0.83）、0.767（0.75-0.79）和 0.785（0.77-0.80）。WPS 的 AUPRC 为 0.517（0.50-0.53），REMS 为 0.547（0.53-0.56），MEWS 为 0.445（0.42-0.46），APACHE II 为 0.630（0.61-0.65），SAPS II 为 0.559（0.54-0.58），SOFA 为 0.564（0.54-0.57）。除 MEWS 和 SOFA 外，所有模型的校准都较好。最准确的模型属于 APACHE II，其 BS 最低。
APACHE II 优于所有 ED 和 ICU 模型，在区分度、校准和预测概率准确性方面，是预测 ICU 患者院内死亡率最准确的模型。除 MEWS 外，其余模型的区分度均较好，部分模型校准度较好。有趣的是，尽管 REMS 比 SAPS II 简单，但这两个模型的表现相似。临床医生可以利用 REMS 作为更全面临床评估的一部分，以更有效地管理患者。
