Danopoulos Evangelos, Aston John A D, Shah Aarushi, Schneider Claudia R
Statistical Laboratory, Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, Cambridge, UK.
School of Psychology, Speech and Hearing, University of Canterbury, Christchurch, New Zealand.
Syst Rev. 2025 Jan 13;14(1):10. doi: 10.1186/s13643-025-02757-8.
Scientific papers increasingly put forward scientific-based policy recommendations (SPRs) as a means of closing the circle of science, policy and practice. Assessing the quality of such SPRs is crucial, especially within the context of a systematic review. Here, we present ECR-P (Evidence Communication Rules for Policy)-a critical appraisal tool that we have developed, which can be used in assessing not only the quality of SPRs but also the quality of their evidence base and how effectively these have both been communicated.
The rationale behind ECR-P centres on three dimensions of quality; two are the well-established concepts of internal and external validity. Here, we introduce a third-evidence communication-encompassing both evidence veracity and quality of communication. Elements of the three dimensions of quality are considered within the context of the five rules of evidence communication. These are as follows: inform, not persuade; offer balance, not false balance; disclose uncertainties; state evidence quality and pre-empt misunderstandings.
Development of ECR-P has been carried out by an interdisciplinary team and was piloted with a systematic review reported more fully elsewhere. ECR-P comprises a set of preliminary considerations which capture key aspects for the assessment, leading on to the main tool whose structure is domain-based, each domain mapping to one of the five rules of evidence communication. The domains include 25 signalling questions designed to obtain essential information for the critical appraisal. The questions focus on either the study's evidence or the policy recommendations. Domain-based judgement is derived from responses to the signalling questions and an accompanying algorithm, followed by an overall quality judgement.
ECR-P has been designed to provide a standardised and transparent approach to assess the quality and communication of SPRs and their evidence base. The tool, which could be applied across all scientific fields, has been developed to fit primarily with the systematic reviewing process but could also serve as a stand-alone tool. Besides review assessors, it can also be used by policymakers, researchers, peer reviewers, editors and any other stakeholders interested in evidence-based policymaking and high-quality evidence communication. We encourage further independent testing of the tool in real-world evidence-based research.
科学论文越来越多地提出基于科学的政策建议（SPR），作为闭合科学、政策和实践循环的一种方式。评估此类SPR的质量至关重要，尤其是在系统评价的背景下。在此，我们介绍ECR-P（政策证据沟通规则）——一种我们开发的批判性评价工具，它不仅可用于评估SPR的质量，还可用于评估其证据基础的质量以及这些内容的沟通效果。
ECR-P背后的基本原理集中在质量的三个维度；其中两个是已确立的内部效度和外部效度概念。在此，我们引入第三个维度——证据沟通，它涵盖证据准确性和沟通质量。质量的三个维度的要素在证据沟通的五条规则的背景下进行考量。具体如下：提供信息，而非说服；保持平衡，而非虚假平衡；披露不确定性；说明证据质量并避免误解。
ECR-P由一个跨学科团队开发，并在一项系统评价中进行了试点，该系统评价在其他地方有更全面的报道。ECR-P包括一组初步考量因素，这些因素抓住了评估的关键方面，进而引出主要工具，其结构基于领域，每个领域对应证据沟通的五条规则之一。这些领域包括25个信号问题，旨在获取进行批判性评价所需的基本信息。这些问题聚焦于研究证据或政策建议。基于领域的判断来自对信号问题的回答及配套算法，随后进行整体质量判断。
ECR-P旨在提供一种标准化、透明的方法，以评估SPR及其证据基础的质量和沟通情况。该工具可应用于所有科学领域，主要是为适应系统评价过程而开发，但也可作为独立工具使用。除了综述评估者，政策制定者、研究人员、同行评审者、编辑以及任何其他对循证决策和高质量证据沟通感兴趣的利益相关者也可使用。我们鼓励在实际的循证研究中对该工具进行进一步的独立测试。
