Department of Medicine, King Faisal Specialist Hospital & Research Centre, Riyadh 11211, Saudi Arabia.
Healthcare Information & Technology Affairs, King Faisal Specialist Hospital & Research Centre, Riyadh 11211, Saudi Arabia.
Int J Environ Res Public Health. 2022 Mar 3;19(5):2958. doi: 10.3390/ijerph19052958.
Clinicians urgently need reliable and stable tools to predict the severity of COVID-19 infection for hospitalized patients to enhance the utilization of hospital resources and supplies. Published COVID-19 related guidelines are frequently being updated, which impacts its utilization as a stable go-to resource for informing clinical and operational decision-making processes. In addition, many COVID-19 patient-level severity prediction tools that were developed during the early stages of the pandemic failed to perform well in the hospital setting due to many challenges including data availability, model generalization, and clinical validation. This study describes the experience of a large tertiary hospital system network in the Middle East in developing a real-time severity prediction tool that can assist clinicians in matching patients with appropriate levels of needed care for better management of limited health care resources during COVID-19 surges. It also provides a new perspective for predicting patients' COVID-19 severity levels at the time of hospital admission using comprehensive data collected during the first year of the pandemic in the hospital. Unlike many previous studies for a similar population in the region, this study evaluated 4 machine learning models using a large training data set of 1386 patients collected between March 2020 and April 2021. The study uses comprehensive COVID-19 patient-level clinical data from the hospital electronic medical records (EMR), vital sign monitoring devices, and Polymerase Chain Reaction (PCR) machines. The data were collected, prepared, and leveraged by a panel of clinical and data experts to develop a multi-class data-driven framework to predict severity levels for COVID-19 infections at admission time. Finally, this study provides results from a prospective validation test conducted by clinical experts in the hospital. The proposed prediction framework shows excellent performance in concurrent validation (n=462 patients, March 2020-April 2021) with highest discrimination obtained with the random forest classification model, achieving a macro- and micro-average area under receiver operating characteristics curve (AUC) of 0.83 and 0.87, respectively. The prospective validation conducted by clinical experts (n=185 patients, April-May 2021) showed a promising overall prediction performance with a recall of 78.4-90.0% and a precision of 75.0-97.8% for different severity classes.
临床医生迫切需要可靠和稳定的工具来预测 COVID-19 感染住院患者的严重程度，以提高医院资源和供应的利用效率。已发布的 COVID-19 相关指南经常更新，这影响了其作为指导临床和运营决策过程的稳定资源的使用。此外，由于数据可用性、模型泛化和临床验证等诸多挑战，许多在大流行早期开发的 COVID-19 患者严重程度预测工具在医院环境中的表现并不理想。本研究描述了一家位于中东的大型三级医院系统网络在开发实时严重程度预测工具方面的经验，该工具可以帮助临床医生根据需要为患者提供适当级别的护理，以在 COVID-19 激增期间更好地管理有限的医疗资源。它还为使用在医院收集的大流行第一年的综合数据在入院时预测患者 COVID-19 严重程度水平提供了新的视角。与该地区类似人群的许多先前研究不同，这项研究使用了 4 种机器学习模型，评估了 2020 年 3 月至 2021 年 4 月期间收集的 1386 名患者的大型训练数据集。该研究使用了来自医院电子病历（EMR）、生命体征监测设备和聚合酶链反应（PCR）机器的综合 COVID-19 患者水平临床数据。数据由一组临床和数据专家收集、准备和利用，以开发一个多类数据驱动的框架，在入院时预测 COVID-19 感染的严重程度。最后，本研究提供了医院临床专家进行的前瞻性验证测试的结果。所提出的预测框架在同期验证中（n=462 名患者，2020 年 3 月至 2021 年 4 月）表现出优异的性能，随机森林分类模型获得了最高的区分度，宏平均和微平均接收者操作特征曲线（AUC）分别为 0.83 和 0.87。临床专家进行的前瞻性验证（n=185 名患者，2021 年 4 月至 5 月）显示出有前途的整体预测性能，不同严重程度类别下的召回率为 78.4-90.0%，精度为 75.0-97.8%。
