McMaster University, Program in Evidence-based Care, Cancer Care Ontario, ON, Canada.
CMAJ. 2010 Jul 13;182(10):1045-52. doi: 10.1503/cmaj.091714. Epub 2010 May 31.
We undertook research to improve the AGREE instrument, a tool used to evaluate guidelines. We tested a new seven-point scale, evaluated the usefulness of the original items in the instrument, investigated evidence to support shorter, tailored versions of the tool, and identified areas for improvement.
We report on one component of a larger study that used a mixed design with four factors (user type, clinical topic, guideline and condition). For the analysis reported in this article, we asked participants to read a guideline and use the AGREE items to evaluate it based on a seven-point scale, to complete three outcome measures related to adoption of the guideline, and to provide feedback on the instrument's usefulness and how to improve it.
Guideline developers gave lower-quality ratings than did clinicians or policy-makers. Five of six domains were significant predictors of participants' outcome measures (p < 0.05). All domains and items were rated as useful by stakeholders (mean scores > 4.0) with no significant differences by user type (p > 0.05). Internal consistency ranged between 0.64 and 0.89. Inter-rater reliability was satisfactory. We received feedback on how to improve the instrument.
Quality ratings of the AGREE domains were significant predictors of outcome measures associated with guideline adoption: guideline endorsements, overall intentions to use guidelines, and overall quality of guidelines. All AGREE items were assessed as useful in determining whether a participant would use a guideline. No clusters of items were found more useful by some users than others. The measurement properties of the seven-point scale were promising. These data contributed to the refinements and release of the AGREE II.
我们进行了研究，以改进 AGREE 工具，该工具用于评估指南。我们测试了一个新的七点量表，评估了工具中原始项目的有用性，研究了支持工具更短、更定制版本的证据，并确定了需要改进的领域。
我们报告了一项更大研究的一个组成部分，该研究采用了混合设计，具有四个因素（用户类型、临床主题、指南和情况）。在本文报告的分析中，我们要求参与者阅读指南，并使用 AGREE 项目根据七点量表对其进行评估，完成与指南采用相关的三个结果测量，并提供有关工具有用性和如何改进工具的反馈。
指南制定者的评分质量低于临床医生或决策者。六个领域中的五个是参与者结果测量的显著预测因素（p < 0.05）。所有领域和项目都被利益相关者评为有用（平均得分> 4.0），用户类型之间没有显着差异（p > 0.05）。内部一致性在 0.64 到 0.89 之间。评分者间可靠性令人满意。我们收到了有关如何改进工具的反馈。
AGREE 领域的质量评分是与指南采用相关的结果测量的显著预测因素：指南认可、总体使用指南的意图以及指南的总体质量。所有 AGREE 项目都被评估为在确定参与者是否使用指南时有用。没有发现某些用户比其他用户更有用的项目集群。七点量表的测量特性很有希望。这些数据为 AGREE II 的改进和发布做出了贡献。
