Center for Quantitative Health, Massachusetts General Hospital, 185 Cambridge Street, Boston, MA 02114, USA; Research Information Science and Computing, Mass General Brigham, 399 Revolution Drive, Somerville, MA 02145, USA.
Center for Quantitative Health, Massachusetts General Hospital, 185 Cambridge Street, Boston, MA 02114, USA.
Gen Hosp Psychiatry. 2022 Jan-Feb;74:9-17. doi: 10.1016/j.genhosppsych.2021.10.005. Epub 2021 Nov 2.
To validate a previously published machine learning model of delirium risk in hospitalized patients with coronavirus disease 2019 (COVID-19).
Using data from six hospitals across two academic medical networks covering care occurring after initial model development, we calculated the predicted risk of delirium using a previously developed risk model applied to diagnostic, medication, laboratory, and other clinical features available in the electronic health record (EHR) at time of hospital admission. We evaluated the accuracy of these predictions against subsequent delirium diagnoses during that admission.
Of the 5102 patients in this cohort, 716 (14%) developed delirium. The model's risk predictions produced a c-index of 0.75 (95% CI, 0.73-0.77) with 27.7% of cases occurring in the top decile of predicted risk scores. Model calibration was diminished compared to the initial COVID-19 wave.
This EHR delirium risk prediction model, developed during the initial surge of COVID-19 patients, produced consistent discrimination over subsequent larger waves; however, with changing cohort composition and delirium occurrence rates, model calibration decreased. These results underscore the importance of calibration, and the challenge of developing risk models for clinical contexts where standard of care and clinical populations may shift.
验证先前发表的用于预测 2019 冠状病毒病（COVID-19）住院患者发生谵妄风险的机器学习模型。
利用来自两个学术医疗网络的六家医院的数据，这些数据涵盖了初始模型开发后发生的护理情况，我们使用先前开发的风险模型，根据入院时电子病历（EHR）中可获得的诊断、用药、实验室和其他临床特征，计算谵妄风险的预测值。我们根据该入院期间的后续谵妄诊断评估这些预测值的准确性。
在该队列的 5102 名患者中，716 名（14%）发生了谵妄。该模型的风险预测产生了 0.75 的 c 指数（95%CI，0.73-0.77），有 27.7%的病例发生在预测风险评分最高的十分位数中。与最初的 COVID-19 浪潮相比，模型校准有所降低。
该 EHR 谵妄风险预测模型是在 COVID-19 患者最初激增期间开发的，在随后更大的浪潮中保持了一致的区分度；然而，随着队列构成和谵妄发生率的变化，模型校准有所下降。这些结果强调了校准的重要性，以及在标准治疗和临床人群可能发生变化的临床环境中开发风险模型的挑战。
