Department of Social Policy and Intervention, University of Oxford, Oxford, OX1 2ER, UK.
London School of Hygiene and Tropical Medicine, London, WC1E 7HT, UK.
Res Synth Methods. 2018 Jun;9(2):224-242. doi: 10.1002/jrsm.1290. Epub 2018 Mar 2.
Rating the quality of a body of evidence is an increasingly common component of research syntheses on intervention effectiveness. This study sought to identify and examine existing systems for rating the quality of a body of evidence on the effectiveness of health and social interventions.
We used a multicomponent search strategy to search for full-length reports of systems for rating the quality of a body of evidence on the effectiveness of health and social interventions published in English from 1995 onward. Two independent reviewers extracted data from each eligible system on the evidence domains included, as well as the development and dissemination processes for each system.
Seventeen systems met our eligibility criteria. Across systems, we identified 13 discrete evidence domains: study design, study execution, consistency, measures of precision, directness, publication bias, magnitude of effect, dose-response, plausible confounding, analogy, robustness, applicability, and coherence. We found little reporting of rigorous procedures in the development and dissemination of evidence rating systems.
We identified 17 systems for rating the quality of a body of evidence on intervention effectiveness across health and social policy. Existing systems vary greatly in the domains they include and how they operationalize domains, and most have important limitations in their development and dissemination. The construct of the quality of the body of evidence was defined in a few systems largely extending the Grading of Recommendations Assessment, Development, and Evaluation approach. Grading of Recommendations Assessment, Development, and Evaluation was found to be unique in its comprehensive guidance, rigorous development, and dissemination strategy.
对证据质量进行评估是干预效果研究综合分析中越来越常见的组成部分。本研究旨在识别和检查现有的评估健康和社会干预效果的证据质量的系统。
我们使用多组分搜索策略，搜索自 1995 年以来以英文发表的关于评估健康和社会干预效果的证据质量的系统的全文报告。两名独立的评审员从每个合格的系统中提取关于纳入证据领域的数据，以及每个系统的开发和传播过程。
有 17 个系统符合我们的入选标准。在这些系统中，我们确定了 13 个离散的证据领域：研究设计、研究执行、一致性、精度测量、直接性、发表偏倚、效应大小、剂量反应、似然混杂、类比、稳健性、适用性和连贯性。我们发现，证据评级系统的开发和传播过程中很少有严格程序的报告。
我们在健康和社会政策领域识别出了 17 个用于评估干预效果的证据质量的系统。现有的系统在它们所包含的领域以及如何操作这些领域方面存在很大差异，而且大多数在开发和传播方面都存在重要的局限性。在少数系统中，证据质量的构建被定义为扩展推荐评估、制定和评价方法。推荐评估、制定和评价方法在其全面指导、严格开发和传播策略方面具有独特性。
