Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK
Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK.
BMJ. 2022 May 18;377:e070904. doi: 10.1136/bmj-2022-070904.
A growing number of artificial intelligence (AI)-based clinical decision support systems are showing promising performance in preclinical, in silico, evaluation, but few have yet demonstrated real benefit to patient care. Early stage clinical evaluation is important to assess an AI system’s actual clinical performance at small scale, ensure its safety, evaluate the human factors surrounding its use, and pave the way to further large scale trials. However, the reporting of these early studies remains inadequate. The present statement provides a multistakeholder, consensus-based reporting guideline for the Developmental and Exploratory Clinical Investigations of DEcision support systems driven by Artificial Intelligence (DECIDE-AI). We conducted a two round, modified Delphi process to collect and analyse expert opinion on the reporting of early clinical evaluation of AI systems. Experts were recruited from 20 predefined stakeholder categories. The final composition and wording of the guideline was determined at a virtual consensus meeting. The checklist and the Explanation & Elaboration (E&E) sections were refined based on feedback from a qualitative evaluation process. 123 experts participated in the first round of Delphi, 138 in the second, 16 in the consensus meeting, and 16 in the qualitative evaluation. The DECIDE-AI reporting guideline comprises 17 AI specific reporting items (made of 28 subitems) and 10 generic reporting items, with an E&E paragraph provided for each. Through consultation and consensus with a range of stakeholders, we have developed a guideline comprising key items that should be reported in early stage clinical studies of AI-based decision support systems in healthcare. By providing an actionable checklist of minimal reporting items, the DECIDE-AI guideline will facilitate the appraisal of these studies and replicability of their findings.
越来越多基于人工智能（AI）的临床决策支持系统在临床前和计算机模拟评估中表现出有前景的性能，但很少有系统能真正为患者护理带来益处。早期临床评估对于评估 AI 系统在小规模下的实际临床性能、确保其安全性、评估其使用的人为因素以及为进一步的大规模试验铺平道路都很重要。然而，这些早期研究的报告仍然不足。本报告提供了一个由多方利益相关者共同制定的、基于共识的报告指南，用于指导基于人工智能的决策支持系统的发展和探索性临床研究（DECIDE-AI）。我们通过两轮修改后的 Delphi 流程，收集和分析了专家对早期 AI 系统临床评估报告的意见。专家来自 20 个预先定义的利益相关者类别中招募。指南的最终组成和措辞在一次虚拟共识会议上确定。清单和说明与解释（E&E）部分根据定性评估过程的反馈进行了细化。123 名专家参加了第一轮 Delphi，138 名参加了第二轮，16 名参加了共识会议，16 名参加了定性评估。DECIDE-AI 报告指南包括 17 个 AI 特定的报告项目（由 28 个子项目组成）和 10 个通用报告项目，每个项目都有一个 E&E 段落。通过与一系列利益相关者的协商和共识，我们制定了一个指南，其中包括在医疗保健中基于 AI 的决策支持系统早期临床研究中应报告的关键项目。通过提供一个最小报告项目的可操作清单，DECIDE-AI 指南将有助于评估这些研究并复制其发现。
