Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada.
Cardiovascular Research Methods Centre, University of Ottawa Heart Institute, Ottawa, Ontario, Canada.
BMC Med Res Methodol. 2021 Dec 20;21(1):285. doi: 10.1186/s12874-021-01451-2.
Systematic reviews are the cornerstone of evidence-based medicine. However, systematic reviews are time consuming and there is growing demand to produce evidence more quickly, while maintaining robust methods. In recent years, artificial intelligence and active-machine learning (AML) have been implemented into several SR software applications. As some of the barriers to adoption of new technologies are the challenges in set-up and how best to use these technologies, we have provided different situations and considerations for knowledge synthesis teams to consider when using artificial intelligence and AML for title and abstract screening.
We retrospectively evaluated the implementation and performance of AML across a set of ten historically completed systematic reviews. Based upon the findings from this work and in consideration of the barriers we have encountered and navigated during the past 24 months in using these tools prospectively in our research, we discussed and developed a series of practical recommendations for research teams to consider in seeking to implement AML tools for citation screening into their workflow.
We developed a seven-step framework and provide guidance for when and how to integrate artificial intelligence and AML into the title and abstract screening process. Steps include: (1) Consulting with Knowledge user/Expert Panel; (2) Developing the search strategy; (3) Preparing your review team; (4) Preparing your database; (5) Building the initial training set; (6) Ongoing screening; and (7) Truncating screening. During Step 6 and/or 7, you may also choose to optimize your team, by shifting some members to other review stages (e.g., full-text screening, data extraction).
Artificial intelligence and, more specifically, AML are well-developed tools for title and abstract screening and can be integrated into the screening process in several ways. Regardless of the method chosen, transparent reporting of these methods is critical for future studies evaluating artificial intelligence and AML.
系统评价是循证医学的基石。然而，系统评价耗时耗力，人们越来越希望更快地生成证据，同时保持稳健的方法。近年来，人工智能和主动机器学习（AML）已被应用于一些 SR 软件应用程序中。由于采用新技术的一些障碍是设置方面的挑战以及如何最好地使用这些技术，因此我们为知识综合团队提供了不同的情况和考虑因素，以便在使用人工智能和 AML 进行标题和摘要筛选时参考。
我们回顾性地评估了在一组十个历史上完成的系统评价中 AML 的实施和性能。基于这项工作的发现，并考虑到我们在过去 24 个月中在前瞻性地使用这些工具时遇到的障碍以及我们如何克服这些障碍，我们讨论并为研究团队制定了一系列实用建议，以供他们考虑在将 AML 工具集成到他们的工作流程中用于引文筛选时参考。
我们开发了一个七步框架，并提供了何时以及如何将人工智能和 AML 整合到标题和摘要筛选过程中的指南。步骤包括：（1）咨询知识用户/专家小组；（2）制定搜索策略；（3）准备审查团队；（4）准备数据库；（5）构建初始训练集；（6）持续筛选；以及（7）截断筛选。在第 6 步和/或第 7 步中，您还可以通过将一些成员转移到其他审查阶段（例如，全文筛选、数据提取）来优化您的团队。
人工智能，更具体地说，AML 是标题和摘要筛选的成熟工具，可以通过多种方式整合到筛选过程中。无论选择哪种方法，对这些方法进行透明报告对于未来评估人工智能和 AML 的研究至关重要。
