Department of Computer Science, Stanford University, Stanford, CA, USA.
Center for Biomedical Informatics Research, Stanford University School of Medicine, Stanford, CA, USA; Department of Surgery, Division of Vascular Surgery, Stanford University School of Medicine, Stanford, CA, USA.
J Biomed Inform. 2023 Mar;139:104319. doi: 10.1016/j.jbi.2023.104319. Epub 2023 Feb 13.
Despite the creation of thousands of machine learning (ML) models, the promise of improving patient care with ML remains largely unrealized. Adoption into clinical practice is lagging, in large part due to disconnects between how ML practitioners evaluate models and what is required for their successful integration into care delivery. Models are just one component of care delivery workflows whose constraints determine clinicians' abilities to act on models' outputs. However, methods to evaluate the usefulness of models in the context of their corresponding workflows are currently limited. To bridge this gap we developed APLUS, a reusable framework for quantitatively assessing via simulation the utility gained from integrating a model into a clinical workflow. We describe the APLUS simulation engine and workflow specification language, and apply it to evaluate a novel ML-based screening pathway for detecting peripheral artery disease at Stanford Health Care.
尽管创建了数千个机器学习 (ML) 模型，但通过 ML 改善患者护理的承诺在很大程度上仍未实现。ML 在临床实践中的采用滞后，很大程度上是因为 ML 从业者评估模型的方式与模型成功整合到护理提供中的要求之间存在脱节。模型只是护理提供工作流程的一个组成部分，其约束决定了临床医生根据模型输出采取行动的能力。然而，目前评估模型在其相应工作流程中的有用性的方法有限。为了弥合这一差距，我们开发了 APLUS，这是一个可重复使用的框架，用于通过模拟评估将模型集成到临床工作流程中所获得的效用。我们描述了 APLUS 模拟引擎和工作流程规范语言，并将其应用于评估斯坦福健康保健中心用于检测外周动脉疾病的新型基于 ML 的筛查途径。
