J Med Libr Assoc. 2021 Oct 1;109(4):599-608. doi: 10.5195/jmla.2021.1219.
The aim of this project was to validate search filters for systematic reviews, intervention studies, and observational studies translated from Ovid MEDLINE and Embase syntax and used for searches in PubMed and Embase.com during the development of evidence summaries supporting first aid guidelines. We aimed to achieve a balance among recall, specificity, precision, and number needed to read (NNR).
Reference gold standards were constructed per study type derived from existing evidence summaries. Search filter performance was assessed through retrospective searches and measurement of relative recall, specificity, precision, and NNR when using the translated search filters. Where necessary, search filters were optimized. Adapted filters were validated in separate validation gold standards.
Search filters for systematic reviews and observational studies reached recall of ≥85% in both PubMed and Embase. Corresponding specificities for systematic review filters were ≥96% in both databases, with a precision of 9.7% (NNR 10) in PubMed and 5.4% (NNR 19) in Embase. For observational study filters, specificity, precision, and NNR were 68%, 2%, and 51 in PubMed and 47%, 0.8%, and 123 in Embase, respectively. These filters were considered sufficiently effective. Search filters for intervention studies reached a recall of 85% and 83% in PubMed and Embase, respectively. Optimization led to recall of ≥95% with specificity, precision, and NNR of 49%, 1.3%, and 79 in PubMed and 56%, 0.74%, and 136 in Embase, respectively.
We report validated filters to search for systematic reviews, observational studies, and intervention studies in guideline projects in PubMed and Embase.com.
本项目旨在验证从 Ovid MEDLINE 和 Embase 语法翻译而来的、用于在 PubMed 和 Embase.com 中搜索的系统评价、干预研究和观察性研究检索过滤器，这些过滤器在支持急救指南的证据摘要的开发过程中得到了应用。我们旨在实现召回率、特异性、准确性和阅读次数（NNR）之间的平衡。
根据现有证据摘要，针对每种研究类型构建参考金标准。通过回顾性搜索并测量使用翻译后的检索过滤器时的相对召回率、特异性、准确性和 NNR，评估检索过滤器的性能。在必要时，对检索过滤器进行优化。适应性过滤器在单独的验证金标准中进行验证。
系统评价和观察性研究的检索过滤器在 PubMed 和 Embase 中均达到了≥85%的召回率。在两个数据库中，系统评价过滤器的特异性均≥96%，在 PubMed 中的准确性为 9.7%（NNR 为 10），在 Embase 中的准确性为 5.4%（NNR 为 19）。对于观察性研究过滤器，特异性、准确性和 NNR 在 PubMed 中分别为 68%、2%和 51，在 Embase 中分别为 47%、0.8%和 123。这些过滤器被认为是足够有效的。干预研究的检索过滤器在 PubMed 和 Embase 中的召回率分别为 85%和 83%。优化后，在 PubMed 中的召回率≥95%，特异性、准确性和 NNR 分别为 49%、1.3%和 79，在 Embase 中的召回率≥95%，特异性、准确性和 NNR 分别为 56%、0.74%和 136。
我们报告了在 PubMed 和 Embase.com 中针对指南项目检索系统评价、观察性研究和干预研究的经过验证的过滤器。
