Guimarães Nathalia Sernizon, Ferreira Andrêa J F, Ribeiro Silva Rita de Cássia, de Paula Adelzon Assis, Lisboa Cinthia Soares, Magno Laio, Ichiara Maria Yury, Barreto Maurício Lima
Institute of Collective Health. Federal University of Bahia, Salvador, Bahia, Brazil.
Centre for Data and Knowledge Integration for Health (CIDACS), Oswaldo Cruz Foundation, Salvador, Bahia, Brazil; The Ubuntu Center on Racism, Global Movements, and Population Health Equity, Dornsife School of Public Health, Drexel University, Philadelphia, PA, USA.
J Clin Epidemiol. 2022 Dec;152:110-115. doi: 10.1016/j.jclinepi.2022.10.009. Epub 2022 Oct 12.
Here, we examined the accuracy measures of a set of automated deduplication tools to identify duplicate in the eligibility process of systematic reviews.
A planned search strategy was carried out on seven electronic databases until May 31, 2021. Using manual search as the reference standard, we assessed sensibility, specificity, negative predictive value, and positive predictive value (PPV).
Specificity ranged from 0.96 to 1.00. Rayyan, Mendeley, and Systematic Review Accelerator (SRA) presented high sensibility (0.98 [95% CI = 0.94-1.00]; 0.93 [95% CI = 0.88-0.97] and 0.90 [95% CI = 0.84-0.95], respectively), whereas EndNote X9 and Zotero had only fair sensitivity (0.73 [95% CI = 0.65-0.80] and 0.74 [95% CI = 0.66-0.81], respectively). Negative predictive value ranged from 0.99 to 1.00. Mendeley and SRA had good PPV (0.93 [95% CI = 0.88-0.97] and 0.99 [95% CI = 0.96-1.00], respectively). PPV was fair for EndNote X9 (0.61 [95% CI = 0.54-0.69]) and Zotero (0.62 [95% CI = 0.54-0.69]) and poor for Rayyan (0.41 [95% CI = 0.36-0.47]).
Choosing the most suitable tool depends on its interface's characteristics, the algorithm to identify and exclude duplicates, and the transparency of the process. Therefore, Rayyan, Mendeley, and SRA proved to be accurate enough for the systematic reviews' deduplication step.
在此,我们检验了一组自动重复数据删除工具在系统评价的纳入过程中识别重复项的准确性指标。
在2021年5月31日前,对七个电子数据库实施了计划好的检索策略。以手动检索作为参考标准,我们评估了敏感度、特异度、阴性预测值和阳性预测值(PPV)。
特异度范围为0.96至1.00。Rayyan、Mendeley和系统评价加速器(SRA)表现出高敏感度(分别为0.98[95%CI=0.94-1.00];0.93[95%CI=0.88-0.97]和0.90[95%CI=0.84-0.95]),而EndNote X9和Zotero的敏感度一般(分别为0.73[95%CI=0.65-0.80]和0.74[95%CI=0.66-0.81])。阴性预测值范围为0.99至1.00。Mendeley和SRA的阳性预测值良好(分别为0.93[95%CI=0.88-0.97]和0.99[95%CI=0.96-1.00])。EndNote X9(0.61[95%CI=0.54-0.69])和Zotero(0.62[95%CI=0.54-0.69])的阳性预测值一般,Rayyan的阳性预测值较差(0.41[95%CI=0.36-0.47])。
选择最合适的工具取决于其界面特性、识别和排除重复项的算法以及过程的透明度。因此,Rayyan、Mendeley和SRA被证明在系统评价的重复数据删除步骤中足够准确。