Jin Mauricio F, Noseworthy Peter A, Yao Xiaoxi
Department of Internal Medicine, Mayo Clinic, Rochester, MN.
Department of Cardiovascular Medicine, Mayo Clinic, Rochester, MN.
Mayo Clin Proc Digit Health. 2024 Aug 6;2(4):499-510. doi: 10.1016/j.mcpdig.2024.06.010. eCollection 2024 Dec.
The emergence of artificial intelligence (AI) and other digital solutions in health care has considerably altered the landscape of medical research and patient care. Rigorous evaluation in routine practice settings is fundamental to the ethical use of AI and consists of 3 stages of evaluations: technical performance, usability and acceptability, and health impact evaluation. Pragmatic trials often play a key role in the health impact evaluation. The current review introduces the concept of pragmatic trials, their role in AI evaluation, the challenges of conducting pragmatic trials, and strategies to mitigate the challenges. We also examined common designs used in pragmatic trials and highlighted examples of published or ongoing AI trials. As more health systems advance into learning health systems, where outcomes are continuously evaluated to refine processes and tools, pragmatic trials embedded into everyday practice, leveraging data and infrastructure from delivering health care, will be a critical part of the feedback cycle for learning and improvement.
人工智能（AI）及其他数字解决方案在医疗保健领域的出现，极大地改变了医学研究和患者护理的格局。在常规实践环境中进行严格评估是人工智能合理应用的基础，包括三个评估阶段：技术性能、可用性和可接受性以及健康影响评估。实用试验通常在健康影响评估中发挥关键作用。本综述介绍了实用试验的概念、其在人工智能评估中的作用、开展实用试验的挑战以及应对挑战的策略。我们还研究了实用试验中常用的设计，并重点介绍了已发表或正在进行的人工智能试验的实例。随着越来越多的卫生系统向学习型卫生系统迈进，即不断评估结果以优化流程和工具，嵌入日常实践、利用提供医疗保健过程中的数据和基础设施的实用试验，将成为学习和改进反馈循环的关键组成部分。
