de Hond Anne A H, Leeuwenberg Artuur M, Hooft Lotty, Kant Ilse M J, Nijman Steven W J, van Os Hendrikus J A, Aardoom Jiska J, Debray Thomas P A, Schuit Ewoud, van Smeden Maarten, Reitsma Johannes B, Steyerberg Ewout W, Chavannes Niels H, Moons Karel G M
Department of Information Technology and Digital Innovation, Leiden University Medical Center, Leiden, The Netherlands.
Clinical AI Implementation and Research Lab, Leiden University Medical Center, Leiden, The Netherlands.
NPJ Digit Med. 2022 Jan 10;5(1):2. doi: 10.1038/s41746-021-00549-7.
While the opportunities of ML and AI in healthcare are promising, the growth of complex data-driven prediction models requires careful quality and applicability assessment before they are applied and disseminated in daily practice. This scoping review aimed to identify actionable guidance for those closely involved in AI-based prediction model (AIPM) development, evaluation and implementation including software engineers, data scientists, and healthcare professionals and to identify potential gaps in this guidance. We performed a scoping review of the relevant literature providing guidance or quality criteria regarding the development, evaluation, and implementation of AIPMs using a comprehensive multi-stage screening strategy. PubMed, Web of Science, and the ACM Digital Library were searched, and AI experts were consulted. Topics were extracted from the identified literature and summarized across the six phases at the core of this review: (1) data preparation, (2) AIPM development, (3) AIPM validation, (4) software development, (5) AIPM impact assessment, and (6) AIPM implementation into daily healthcare practice. From 2683 unique hits, 72 relevant guidance documents were identified. Substantial guidance was found for data preparation, AIPM development and AIPM validation (phases 1-3), while later phases clearly have received less attention (software development, impact assessment and implementation) in the scientific literature. The six phases of the AIPM development, evaluation and implementation cycle provide a framework for responsible introduction of AI-based prediction models in healthcare. Additional domain and technology specific research may be necessary and more practical experience with implementing AIPMs is needed to support further guidance.
虽然机器学习和人工智能在医疗保健领域的机会前景广阔，但复杂的数据驱动预测模型的发展在应用于日常实践并推广之前，需要仔细进行质量和适用性评估。本范围综述旨在为那些密切参与基于人工智能的预测模型（AIPM）开发、评估和实施的人员（包括软件工程师、数据科学家和医疗保健专业人员）确定可操作的指导意见，并找出该指导意见中潜在的差距。我们使用全面的多阶段筛选策略，对有关AIPM开发、评估和实施的指导意见或质量标准的相关文献进行了范围综述。检索了PubMed、科学网和美国计算机协会数字图书馆，并咨询了人工智能专家。从已识别的文献中提取主题，并在本综述核心的六个阶段进行总结：（1）数据准备，（2）AIPM开发，（3）AIPM验证，（4）软件开发，（5）AIPM影响评估，以及（6）AIPM在日常医疗实践中的实施。从2683个独特的搜索结果中，识别出72份相关指导文件。在数据准备、AIPM开发和AIPM验证（第1 - 3阶段）方面发现了大量指导意见，而在科学文献中，后期阶段（软件开发、影响评估和实施）显然受到的关注较少。AIPM开发、评估和实施周期的六个阶段为在医疗保健中负责任地引入基于人工智能的预测模型提供了一个框架。可能需要更多特定领域和技术的研究，并且需要更多实施AIPM的实践经验来支持进一步的指导。
