Saelmans Alexander, Seinen Tom, Pera Victor, Markus Aniek F, Fridgeirsson Egill, John Luis H, Schiphof-Godart Lieke, Rijnbeek Peter, Reps Jenna, Williams Ross
Department of Medical Informatics, Erasmus University Medical Center, Rotterdam, Netherlands.
Janssen Research and Development, Titusville, NJ.
Mayo Clin Proc Digit Health. 2025 May 23;3(3):100228. doi: 10.1016/j.mcpdig.2025.100228. eCollection 2025 Sep.
To summarize the implementation approaches and updating methods of clinically implemented models and consecutively advise researchers on the implementation and updating.
We included studies describing the implementation of prognostic binary prediction models in a clinical setting. We retrieved articles from Embase, Medline, and Web of Science from January 1, 2010, to January 1, 2024. We performed data extraction, based on Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis and Prediction Model Risk of Bias Assessment guidelines, and summarized.
The search yielded 1872 articles. Following screening, 37 articles, describing 56 prediction models, were eligible for inclusion. The overall risk of bias was high in 86% of publications. In model development and internal validation, 32% of the models was assessed for calibration. External validation was performed for 27% of the models. Most models were implemented into the hospital information system (63%), followed by a web application (32%) and a patient decision aid tool (5%). Moreover, 13% of models have been updated following implementation.
Impact assessments generally showed successful model implementation and the ability to improve patient care, despite not fully adhering to prediction modeling best practice. Both impact assessment and updating could play a key role in identifying and lowering bias in models.
总结临床实施模型的实施方法和更新方法，并相继为研究人员提供有关实施和更新的建议。
我们纳入了描述预后二元预测模型在临床环境中实施情况的研究。我们检索了2010年1月1日至2024年1月1日期间Embase、Medline和科学网的文章。我们根据个体预后或诊断多变量预测模型的透明报告以及预测模型偏倚评估风险指南进行数据提取并总结。
检索得到1872篇文章。经过筛选，37篇描述56个预测模型的文章符合纳入标准。86%的出版物总体偏倚风险较高。在模型开发和内部验证中，32%的模型进行了校准评估。27%的模型进行了外部验证。大多数模型被应用于医院信息系统（63%），其次是网络应用程序（32%）和患者决策辅助工具（5%）。此外，13%的模型在实施后进行了更新。
尽管没有完全遵循预测建模的最佳实践，但影响评估总体上显示模型实施成功且有改善患者护理的能力。影响评估和更新在识别和降低模型偏差方面都可能发挥关键作用。
