Marshall Iain J, Trikalinos Thomas A, Soboczenski Frank, Yun Hye Sun, Kell Gregory, Marshall Rachel, Wallace Byron C
School of Life Course and Population Sciences, King's College London, London, UK.
Center for Evidence Synthesis in Health, Brown University, Providence, RI, USA.
J Clin Epidemiol. 2023 Jan;153:26-33. doi: 10.1016/j.jclinepi.2022.08.013. Epub 2022 Sep 20.
The aim of this study is to describe and pilot a novel method for continuously identifying newly published trials relevant to a systematic review, enabled by combining artificial intelligence (AI) with human expertise.
We used RobotReviewer LIVE to keep a review of COVID-19 vaccination trials updated from February to August 2021. We compared the papers identified by the system with those found by the conventional manual process by the review team.
The manual update searches (last search date July 2021) retrieved 135 abstracts, of which 31 were included after screening (23% precision, 100% recall). By the same date, the automated system retrieved 56 abstracts, of which 31 were included after manual screening (55% precision, 100% recall). Key limitations of the system include that it is limited to searches of PubMed/MEDLINE, and considers only randomized controlled trial reports. We aim to address these limitations in future. The system is available as open-source software for further piloting and evaluation.
Our system identified all relevant studies, reduced manual screening work, and enabled rolling updates on publication of new primary research.
本研究旨在描述并试行一种将人工智能（AI）与人类专业知识相结合的新方法，用于持续识别与系统评价相关的新发表试验。
我们使用“实时机器人审阅者”（RobotReviewer LIVE）对2021年2月至8月的新冠疫苗接种试验进行更新审查。我们将系统识别出的论文与审查团队通过传统手动流程找到的论文进行了比较。
手动更新检索（最后检索日期为2021年7月）获取了135篇摘要，其中31篇经筛选后被纳入（精确率23%，召回率100%）。截至同一日期，自动化系统检索到56篇摘要，其中31篇经人工筛选后被纳入（精确率55%，召回率100%）。该系统的主要局限性包括仅限于检索PubMed/MEDLINE，且仅考虑随机对照试验报告。我们计划在未来解决这些局限性。该系统作为开源软件可供进一步试行和评估。
我们的系统识别出了所有相关研究，减少了人工筛选工作，并能够在新的原始研究发表时进行滚动更新。
