Salehinejad Hojjat, Meehan Anne M, Rahman Parvez A, Core Marcia A, Borah Bijan J, Caraballo Pedro J
Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA.
Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA.
EClinicalMedicine. 2023 Nov 16;66:102312. doi: 10.1016/j.eclinm.2023.102312. eCollection 2023 Dec.
Threshold-based early warning systems (EWS) are used to predict adverse events (Aes). Machine learning (ML) algorithms that incorporate all EWS scores prior to an event may perform better in hospitalized patients.
The deterioration index (DI) is a proprietary EWS. A threshold of DI >60 is used to predict a composite AE: all-cause mortality, cardiac arrest, transfer to intensive care, and evaluation by the rapid response team in practice. The DI scores were collected for adult patients (≥18 y-o) hospitalized on medical or surgical services during 8-23-2021 to 3-31-2022 from four different Mayo Clinic sites in the United States. A novel ML model was developed and trained on a retrospective cohort of hospital encounters. DI scores were represented in a high-dimensional space using random convolution kernels to facilitate training of a classifier and the area under the receiver operator characteristics curve (AUC) was calculated. Multiple time intervals prior to an AE were analyzed. A leave-one-out cross-validation protocol was used to evaluate performance across separate clinic sites.
Three different classifiers were trained on 59,617 encounter-derived DI scores in high-dimensional feature space and the AUCs were compared to two threshold models. All three tested classifiers improved the AUC over the threshold approaches from 0.56 and 0.57 to 0.76, 0.85 and 0.94. Time interval analysis of the top performing classifier showed best accuracy in the hour before an event occurred (AUC 0.91), but prediction held up even in the 12 h before an AE (AUC 0.80 at minus 12 h, 0.81 at minus 9 h, 0.85 at minus 6 h, and 0.88 at minus 3 h before an AE). Multisite cross-validation using leave-one-out approach on data from four different clinical sites showed broad generalization performance of the top performing ML model with AUC of 0.91, 0.91, 0.95, and 0.91.
A novel ML model that incorporates all the longitudinal DI scores prior to an AE in a hospitalized patient performs better at outcome prediction than the currently used threshold model. The use of clinical data, a generalized ML technique, and successful multisite cross-validation demonstrate the feasibility of our model in clinical implementation.
No funding to report.
基于阈值的早期预警系统（EWS）用于预测不良事件（AE）。在事件发生前纳入所有EWS评分的机器学习（ML）算法可能在住院患者中表现更佳。
恶化指数（DI）是一种专有的EWS。在实际应用中，DI>60的阈值用于预测复合AE：全因死亡率、心脏骤停、转入重症监护以及由快速反应团队进行评估。收集了2021年8月23日至2022年3月31日期间在美国梅奥诊所四个不同地点接受医疗或外科服务住院的成年患者（≥18岁）的DI评分。开发了一种新型ML模型，并在回顾性队列的医院就诊病例上进行训练。使用随机卷积核在高维空间中表示DI评分，以促进分类器的训练，并计算接收器操作特征曲线（AUC）下的面积。分析了AE发生前的多个时间间隔。采用留一法交叉验证方案评估不同临床地点的性能。
在高维特征空间中，对59,617个源自就诊病例的DI评分训练了三种不同的分类器，并将AUC与两种阈值模型进行比较。所有三种测试分类器均将AUC从阈值方法的0.56和0.57提高到0.76、0.85和0.94。对表现最佳的分类器进行时间间隔分析显示，在事件发生前一小时准确率最高（AUC 0.91），但即使在AE发生前12小时预测效果依然良好（AE前12小时AUC为0.80，前9小时为0.81，前6小时为0.85，前3小时为0.88）。对来自四个不同临床地点的数据采用留一法进行多地点交叉验证，结果显示表现最佳的ML模型具有广泛的泛化性能，AUC分别为0.91、0.91、0.95和0.91。
一种新型ML模型，在住院患者AE发生前纳入所有纵向DI评分，在结局预测方面比目前使用的阈值模型表现更佳。临床数据的使用、通用的ML技术以及成功的多地点交叉验证证明了我们模型在临床实施中的可行性。
无资金报告。
