Afshar Majid, Resnik Felice, Ryan Baumann Mary, Hintzke Josie, Sullivan Anne Gravel, Shah Tina, Stordalen Anthony, Oberst Michael, Dambach Jason, Mrotek Leigh Ann, Quinn Mariah, Abramson Kirsten, Kleinschmidt Peter, Brazelton Tom, Twedt Heidi, Kunstman David, Long John, Patterson Brian, Liao Frank, Rasmussen Stacy, Burnside Elizabeth, Goswami Cherodeep, Gordon Joel
medRxiv. 2025 Jan 21:2024.12.27.24319685. doi: 10.1101/2024.12.27.24319685.
Ambient artificial intelligence offers promise for improving documentation efficiency and reducing provider burden through clinical note generation. However, challenges persist in workflow integration, compliance, and widespread adoption. This study leveraged a Learning Health System (LHS) framework to align research and operations using a hybrid effectiveness-implementation protocol, embedded as pragmatic trial operations within the electronic health record (EHR).
An alpha phase was conducted to pilot technical integration, refine workflows, and determine sample size in planning for a beta phase designed as a pragmatic randomized controlled trial with the Stanford Professional Fulfillment Index (PFI) as primary outcome. During alpha, bi-directional governance was established between IS operations and LHS team with multidisciplinary workgroups for analytics, technical, documentation, and user experience. Ambient AI was embedded into the EHR using Fast Healthcare Interoperability Resources (FHIR), with real-time data dashboards tracking utilization and documentation accuracy for operations and research. Performance metrics were monitored serially using a difference-in-differences (DiD) analysis to detect drift caused by software workflow changes.
The alpha phase, designed as Type 1 Hybrid, informed a 24-week beta phase stepped-wedge trial with 90% power to detect changes in PFI. Across the alpha phase, the weighted median of average provider Ambient AI utilization was 65.4% following Plan-Do-Study-Act cycles addressing organizational feasibility and task-dependent adoption. During initial implementation, a workflow issue caused discrepancies between ICD-10 diagnosis entries and note content, reducing accuracy from 79% to 35% (p < 0.01). After implementing a new note template and provider training, accuracy returned to pre-intervention levels. DiD did not detect significant drifts in work outside of work or time in notes two weeks before and after the new note template. Beta phase enrollment achieved its targeted 66 providers across eight specialties, initiating on schedule.
We provide a novel playbook for integrating Generative AI platforms in healthcare, combining pragmatic trial operations, human-centered design, and real-time monitoring to advance evidence-based implementation.
NCT06517082.
环境人工智能有望通过生成临床记录来提高文档记录效率并减轻医疗服务提供者的负担。然而，在工作流程整合、合规性和广泛采用方面仍存在挑战。本研究利用学习健康系统（LHS）框架，通过混合有效性-实施协议使研究与运营保持一致，并将其作为实用试验运营嵌入电子健康记录（EHR）中。
进行了一个alpha阶段，以试点技术整合、完善工作流程并确定样本量，为设计为实用随机对照试验的beta阶段做准备，该试验以斯坦福专业成就指数（PFI）作为主要结果。在alpha阶段，信息系统运营部门与LHS团队之间建立了双向治理机制，并成立了多学科工作组，负责分析、技术、文档记录和用户体验。利用快速医疗保健互操作性资源（FHIR）将环境人工智能嵌入电子健康记录中，并通过实时数据仪表板跟踪运营和研究的使用情况及文档记录准确性。使用差分分析（DiD）对性能指标进行连续监测，以检测软件工作流程变化引起的偏差。
设计为1型混合试验的alpha阶段为为期24周的beta阶段阶梯楔形试验提供参考，该试验有90%的把握度检测PFI的变化。在整个alpha阶段，在针对组织可行性和任务相关采用情况的计划-执行-研究-行动周期之后，医疗服务提供者环境人工智能平均使用率的加权中位数为65.4%。在最初实施期间，一个工作流程问题导致ICD-10诊断条目与记录内容之间出现差异，准确性从79%降至35%（p<0.01）。在实施新的记录模板并对医疗服务提供者进行培训后，准确性恢复到干预前水平。DiD未检测到新记录模板前后两周笔记中工作时间或工作之外的显著偏差。beta阶段招募了来自八个专业的66名目标医疗服务提供者，按计划启动。
我们提供了一个在医疗保健领域整合生成式人工智能平台 的新颖方案，结合实用试验运营、以人为本的设计和实时监测，以推进基于证据的实施过程。
NCT06517082
