Yun Hyoungju, Choi Jinwook, Park Jeong Ho
Interdisciplinary Program of Medical Informatics, College of Medicine, Seoul National University, Seoul, Republic of Korea.
Department of Biomedical Engineering, College of Medicine, Seoul National University, Seoul, Republic of Korea.
JMIR Med Inform. 2021 Sep 20;9(9):e30770. doi: 10.2196/30770.
The emergency department (ED) triage system to classify and prioritize patients from high risk to less urgent continues to be a challenge.
This study, comprising 80,433 patients, aims to develop a machine learning algorithm prediction model of critical care outcomes for adult patients using information collected during ED triage and compare the performance with that of the baseline model using the Korean Triage and Acuity Scale (KTAS).
To predict the need for critical care, we used 13 predictors from triage information: age, gender, mode of ED arrival, the time interval between onset and ED arrival, reason of ED visit, chief complaints, systolic blood pressure, diastolic blood pressure, pulse rate, respiratory rate, body temperature, oxygen saturation, and level of consciousness. The baseline model with KTAS was developed using logistic regression, and the machine learning model with 13 variables was generated using extreme gradient boosting (XGB) and deep neural network (DNN) algorithms. The discrimination was measured by the area under the receiver operating characteristic (AUROC) curve. The ability of calibration with Hosmer-Lemeshow test and reclassification with net reclassification index were evaluated. The calibration plot and partial dependence plot were used in the analysis.
The AUROC of the model with the full set of variables (0.833-0.861) was better than that of the baseline model (0.796). The XGB model of AUROC 0.861 (95% CI 0.848-0.874) showed a higher discriminative performance than the DNN model of 0.833 (95% CI 0.819-0.848). The XGB and DNN models proved better reclassification than the baseline model with a positive net reclassification index. The XGB models were well-calibrated (Hosmer-Lemeshow test; P>.05); however, the DNN showed poor calibration power (Hosmer-Lemeshow test; P<.001). We further interpreted the nonlinear association between variables and critical care prediction.
Our study demonstrated that the performance of the XGB model using initial information at ED triage for predicting patients in need of critical care outperformed the conventional model with KTAS.
急诊科（ED）分诊系统用于对患者进行分类并按优先级从高风险到低紧急程度排序，这仍然是一项挑战。
本研究纳入80433例患者，旨在利用急诊分诊期间收集的信息开发一种针对成年患者重症监护结局的机器学习算法预测模型，并将其性能与使用韩国分诊及 acuity 量表（KTAS）的基线模型进行比较。
为预测重症监护需求，我们使用了分诊信息中的13个预测指标：年龄、性别、到达急诊科的方式、发病至到达急诊科的时间间隔、就诊原因、主要症状、收缩压、舒张压、脉搏率、呼吸率、体温、血氧饱和度和意识水平。使用逻辑回归建立了基于KTAS的基线模型，并使用极端梯度提升（XGB）和深度神经网络（DNN）算法生成了包含13个变量的机器学习模型。通过受试者操作特征（AUROC）曲线下面积来衡量区分度。评估了用Hosmer-Lemeshow检验进行校准的能力以及用净重新分类指数进行重新分类的能力。分析中使用了校准图和部分依赖图。
包含全套变量的模型的AUROC（0.833 - 0.861）优于基线模型（0.796）。AUROC为0.861（95%CI 0.848 - 0.874）的XGB模型显示出比AUROC为0.833（95%CI 0.819 - 0.848）的DNN模型更高的区分性能。XGB和DNN模型经证明比具有正净重新分类指数的基线模型有更好的重新分类效果。XGB模型校准良好（Hosmer-Lemeshow检验；P>0.05）；然而，DNN显示出较差的校准能力（Hosmer-Lemeshow检验；P<0.001）。我们进一步解释了变量与重症监护预测之间的非线性关联。
我们的研究表明，使用急诊分诊初始信息的XGB模型在预测需要重症监护的患者方面的性能优于传统的KTAS模型。
