Department of Medical Informatics, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran.
Department of Health Information Technology, Faculty of Paramedical Sciences, Mashhad University of Medical Sciences, Mashhad, Iran.
BMC Emerg Med. 2021 Jun 10;21(1):68. doi: 10.1186/s12873-021-00459-7.
Medical scoring systems are potentially useful to make optimal use of available resources. A variety of models have been developed for illness measurement and stratification of patients in Emergency Departments (EDs). This study was aimed to compare the predictive performance of the following six scoring systems: Simple Clinical Score (SCS), Worthing physiological Score (WPS), Rapid Acute Physiology Score (RAPS), Rapid Emergency Medicine Score (REMS), Modified Early Warning Score (MEWS), and Routine Laboratory Data (RLD) to predict in-hospital mortality.
A prospective single-center observational study was conducted from March 2016 to March 2017 in Edalatian ED in Emam Reza Hospital, located in the northeast of Iran. All variables needed to calculate the models were recorded at the time of admission and logistic regression was used to develop the models' prediction probabilities. The Area Under the Curve for Receiver Operating Characteristic (AUC-ROC) and Precision-Recall curves (AUC-PR), Brier Score (BS), and calibration plots were used to assess the models' performance. Internal validation was obtained by 1000 bootstrap samples. Pairwise comparison of AUC-ROC was based on the DeLong test.
A total of 2205 patients participated in this study with a mean age of 61.8 ± 18.5 years. About 19% of the patients died in the hospital. Approximately 53% of the participants were male. The discrimination ability of SCS, WPS, RAPS, REMS, MEWS, and RLD methods were 0.714, 0.727, 0.661, 0.678, 0.698, and 0.656, respectively. Additionally, the AUC-PR of SCS, WPS, RAPS, REMS, EWS, and RLD were 0.39, 0.42, 0.35, 0.34, 0.36, and 0.33 respectively. Moreover, BS was 0.1459 for SCS, 0.1713 for WPS, 0.0908 for RAPS, 0.1044 for REMS, 0.1158 for MEWS, and 0.073 for RLD. Results of pairwise comparison which was performed for all models revealed that there was no significant difference between the SCS and WPS. The calibration plots demonstrated a relatively good concordance between the actual and predicted probability of non-survival for the SCS and WPS models.
Both SCS and WPS demonstrated fair discrimination and good calibration, which were superior to the other models. Further recalibration is however still required to improve the predictive performance of all available models and their use in clinical practice is still unwarranted.
医学评分系统可用于优化资源利用。已开发出多种模型来测量疾病并对急诊科（ED）的患者进行分层。本研究旨在比较以下六种评分系统的预测性能：简单临床评分（SCS）、沃辛生理评分（WPS）、快速急性生理评分（RAPS）、快速急诊医学评分（REMS）、改良早期预警评分（MEWS）和常规实验室数据（RLD），以预测院内死亡率。
这是一项前瞻性单中心观察性研究，于 2016 年 3 月至 2017 年 3 月在位于伊朗东北部埃马米雷扎医院的埃达利安 ED 进行。入院时记录了计算模型所需的所有变量，并使用逻辑回归来开发模型的预测概率。接收者操作特征曲线下的面积（AUC-ROC）和精度-召回曲线（AUC-PR）、Brier 评分（BS）和校准图用于评估模型性能。通过 1000 次 bootstrap 样本进行内部验证。AUC-ROC 的两两比较基于 DeLong 检验。
共有 2205 名患者参加了这项研究，平均年龄为 61.8±18.5 岁。约 19%的患者在医院死亡。大约 53%的参与者为男性。SCS、WPS、RAPS、REMS、MEWS 和 RLD 方法的判别能力分别为 0.714、0.727、0.661、0.678、0.698 和 0.656。此外，SCS、WPS、RAPS、REMS、EWS 和 RLD 的 AUC-PR 分别为 0.39、0.42、0.35、0.34、0.36 和 0.33。此外，SCS 的 BS 为 0.1459，WPS 为 0.1713，RAPS 为 0.0908，REMS 为 0.1044，MEWS 为 0.1158，RLD 为 0.073。对所有模型进行的两两比较结果表明，SCS 和 WPS 之间没有显著差异。校准图表明 SCS 和 WPS 模型的实际和预测非生存概率之间具有较好的一致性。
SCS 和 WPS 均表现出良好的判别力和校准度，优于其他模型。然而，仍需要进一步重新校准以提高所有可用模型的预测性能，并且在临床实践中仍不建议使用。
