基于机器学习的谵妄预测软件在临床常规中的外部验证。
External Validation of a Machine Learning Based Delirium Prediction Software in Clinical Routine.
机构信息
Steiermärkische Krankenanstaltengesellschaft m.b.H, Graz, Austria.
Krankenhaus der Barmherzigen Schwestern Ried, Ried im Innkreis, Austria.
出版信息
Stud Health Technol Inform. 2022 May 16;293:93-100. doi: 10.3233/SHTI220353.
BACKGROUND
Various machine learning (ML) models have been developed for the prediction of clinical outcomes, but there is missing evidence on their performance in clinical routine and external validation.
OBJECTIVES
Our aim was to deploy and prospectively evaluate an already developed delirium prediction software in clinical routine of an external hospital.
METHODS
We compared updated ML models of the software and models re-trained with the external hospital's data. The best models were deployed in clinical routine for one month, and risk predictions for all admitted patients were compared to the risk ratings of a senior physician. After using the software, clinicians completed a questionnaire assessing technology acceptance.
RESULTS
Re-trained models achieved a high discriminative performance (AUROC > 0.92). Compared to clinical risk ratings, the software achieved a sensitivity of 100.0% and a specificity of 90.6%. Usefulness, ease of use and output quality were rated positively by the users.
CONCLUSION
A ML based delirium prediction software achieved a high discriminative performance and high technology acceptance at an external hospital using re-trained ML models.
背景
已经开发出各种机器学习 (ML) 模型来预测临床结果，但在临床常规和外部验证方面缺乏相关证据。
目的
我们的目的是在外部医院的临床常规中部署和前瞻性评估已经开发的谵妄预测软件。
方法
我们比较了软件的更新 ML 模型和使用外部医院数据重新训练的模型。将最佳模型部署在临床常规中一个月，并将所有入院患者的风险预测与高级医师的风险评分进行比较。使用软件后，临床医生完成了一份评估技术接受度的问卷。
结果
重新训练的模型具有较高的判别性能（AUROC>0.92）。与临床风险评分相比，该软件的敏感性为 100.0%，特异性为 90.6%。用户对其有用性、易用性和输出质量评价积极。
结论
使用重新训练的 ML 模型，基于 ML 的谵妄预测软件在外部医院实现了较高的判别性能和较高的技术接受度。
Zotero 插件