Nover Jonathan, Bai Matthew, Tismina Prem, Raut Ganesh, Patel Dhavalkumar, Nadkarni Girish N, Abella Benjamin S, Klang Eyal, Freeman Robert
Department of Emergency Medicine, Mount Sinai Health System, New York, NY.
The Windreich Department of Artificial Intelligence and Human Health, Mount Sinai Medical Center, New York, NY.
Mayo Clin Proc Digit Health. 2025 Jul 9;3(3):100249. doi: 10.1016/j.mcpdig.2025.100249. eCollection 2025 Sep.
To prospectively compare nurse predictions with a machine learning (ML) model for hospital admissions and evaluate whether adding the nurse prediction to ML outputs enhances predictive performance.
In this prospective, observational study at 6 hospitals in a large mixed quaternary/community emergency department (ED) system (annual ED census ∼500,000), triage nurses recorded a binary admission prediction for adult patients. These predictions were compared with an ensemble ML model (XGBoost + Bio-Clinical BERT) trained on structured data (demographics, vital signs, and medical history) and triage text. Nurse predictions were similarly analyzed and then integrated with the ML output to assess for improvement in predictive accuracy.
The ensemble ML model (XGBoost + Bio-Clinical BERT) was trained on 1.8 million historical ED visits (January 2019 to December 2023). It was then tested on 46,912 prospective ED visits with recorded nurse predictions (September 1, 2024 to October 31, 2024). In the prospective arm, nurse predictions yielded an accuracy of 81.6% (95% CI, 81.3-81.9), a sensitivity of 64.8% (63.7-65.8), and a specificity of 85.7% (85.3-86.0). At a 0.30 probability threshold, the ML model attained an accuracy of 85.4% (85.0-85.7) and a sensitivity of 70.8% (69.8-71.7). Combining nurse predictions with the ML output did not improve accuracy beyond the model alone.
Machine learning-based predictions outperformed triage nurse estimates for hospital admissions. These findings suggest that an admission prediction system anchored by ML can perform reliably using data available at triage.
前瞻性比较护士预测与用于医院入院的机器学习（ML）模型，并评估将护士预测添加到ML输出中是否能提高预测性能。
在一个大型综合四级/社区急诊科（ED）系统中的6家医院进行的这项前瞻性观察研究（年度ED普查约500,000人次）中，分诊护士记录了成年患者的二元入院预测。将这些预测与基于结构化数据（人口统计学、生命体征和病史）和分诊文本训练的集成ML模型（XGBoost + 生物临床BERT）进行比较。对护士预测进行类似分析，然后与ML输出整合，以评估预测准确性的提高情况。
集成ML模型（XGBoost + 生物临床BERT）基于180万次历史ED就诊（2019年1月至2023年12月）进行训练。然后在46,912次有记录护士预测的前瞻性ED就诊（2024年9月1日至2024年10月31日）上进行测试。在前瞻性队列中，护士预测的准确率为81.6%（95%CI，81.3 - 81.9），灵敏度为64.8%（63.7 - 65.8），特异度为85.7%（85.3 - 86.0）。在概率阈值为0.30时，ML模型的准确率为85.4%（85.0 - 85.7），灵敏度为70.8%（69.8 - 71.7）。将护士预测与ML输出相结合并没有比单独使用模型提高准确性。
基于机器学习的入院预测优于分诊护士的估计。这些发现表明，以ML为基础的入院预测系统可以利用分诊时可用的数据可靠地运行。
