McMaster University, Juravinski Site, G2 Wing, 1280 Main Street West, Hamilton, ON, L8S 4L8, Canada.
McMaster University, 1280 Main Street West MML-417, Hamilton, ON, L8S 4L8, Canada.
Health Res Policy Syst. 2018 Jun 20;16(1):51. doi: 10.1186/s12961-018-0334-9.
Health systems guidance (HSG) provides recommendations to address health systems challenges. No tools exist to inform HSG developers and users about the components of high quality HSG and to differentiate between HSG of varying quality. In response, we developed a tool to assist with the development, reporting and appraisal of HSG - the Appraisal of Guidelines for Research and Evaluation-Health Systems (AGREE-HS). This paper reports on the validity, usability and initial measurement properties of the AGREE-HS.
To establish face validity (Study 1), stakeholders completed a survey about the AGREE-HS and provided feedback on its content and structure. Revisions to the tool were made in response. To establish usability (Study 2), the revised tool was applied to 85 HSG documents and the appraisers provided feedback about their experiences via an online survey. An initial test of the revised tool's measurement properties, including internal consistency, inter-rater reliability and criterion validity, was conducted. Additional revisions to the tool were made in response.
In Study 1, the AGREE-HS Overview, User Manual, quality item content and structure, and overall assessment questions were rated favourably. Participants indicated that the AGREE-HS would be useful, feasible to use, and that they would apply it in their context. In Study 2, participants indicated that the quality items were easy to understand and apply, and the User Manual, usefulness and usability of the tool were rated favourably. Study 2 participants also indicated intentions to use the AGREE-HS.
The AGREE-HS comprises a User Manual, five quality items and two overall assessment questions. It is available at agreetrust.org.
卫生系统指南（HSG）提供了应对卫生系统挑战的建议。目前尚无工具可以告知 HSG 开发者和使用者高质量 HSG 的组成部分，并区分不同质量的 HSG。有鉴于此，我们开发了一种工具，以协助 HSG 的制定、报告和评估——评估卫生系统研究和评价指南（AGREE-HS）。本文报告了 AGREE-HS 的有效性、可用性和初步测量特性。
为了确立表面有效性（研究 1），利益相关者完成了一项关于 AGREE-HS 的调查，并就其内容和结构提供了反馈。根据反馈对工具进行了修订。为了确立可用性（研究 2），修订后的工具应用于 85 份 HSG 文件，评估者通过在线调查提供了他们的经验反馈。对修订后的工具的初步测量特性进行了测试，包括内部一致性、评分者间可靠性和标准有效性。根据反馈对工具进行了进一步修订。
在研究 1 中，AGREE-HS 概述、用户手册、质量项目内容和结构以及总体评估问题的评分较高。参与者表示，AGREE-HS 将是有用的、可行的，并且将在他们的环境中应用。在研究 2 中，参与者表示质量项目易于理解和应用，用户手册、工具的有用性和可用性得到了较高的评价。研究 2 的参与者也表示有意使用 AGREE-HS。
AGREE-HS 包括用户手册、五个质量项目和两个总体评估问题。它可在 agreetrust.org 获得。
