Brennan Sue E, McKenzie Joanne E, Turner Tari, Redman Sally, Makkar Steve, Williamson Anna, Haynes Abby, Green Sally E
School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia.
Sax Institute, Sydney, Australia.
Health Res Policy Syst. 2017 Jan 17;15(1):1. doi: 10.1186/s12961-016-0162-8.
Capacity building strategies are widely used to increase the use of research in policy development. However, a lack of well-validated measures for policy contexts has hampered efforts to identify priorities for capacity building and to evaluate the impact of strategies. We aimed to address this gap by developing SEER (Seeking, Engaging with and Evaluating Research), a self-report measure of individual policymakers' capacity to engage with and use research.
We used the SPIRIT Action Framework to identify pertinent domains and guide development of items for measuring each domain. Scales covered (1) individual capacity to use research (confidence in using research, value placed on research, individual perceptions of the value their organisation places on research, supporting tools and systems), (2) actions taken to engage with research and researchers, and (3) use of research to inform policy (extent and type of research use). A sample of policymakers engaged in health policy development provided data to examine scale reliability (internal consistency, test-retest) and validity (relation to measures of similar concepts, relation to a measure of intention to use research, internal structure of the individual capacity scales).
Response rates were 55% (150/272 people, 12 agencies) for the validity and internal consistency analyses, and 54% (57/105 people, 9 agencies) for test-retest reliability. The individual capacity scales demonstrated adequate internal consistency reliability (alpha coefficients > 0.7, all four scales) and test-retest reliability (intra-class correlation coefficients > 0.7 for three scales and 0.59 for fourth scale). Scores on individual capacity scales converged as predicted with measures of similar concepts (moderate correlations of > 0.4), and confirmatory factor analysis provided evidence that the scales measured related but distinct concepts. Items in each of these four scales related as predicted to concepts in the measurement model derived from the SPIRIT Action Framework. Evidence about the reliability and validity of the research engagement actions and research use scales was equivocal.
Initial testing of SEER suggests that the four individual capacity scales may be used in policy settings to examine current capacity and identify areas for capacity building. The relation between capacity, research engagement actions and research use requires further investigation.
能力建设策略被广泛用于增加研究在政策制定中的应用。然而，缺乏针对政策背景的有效验证措施阻碍了确定能力建设优先事项以及评估策略影响的努力。我们旨在通过开发SEER（寻求、参与和评估研究）来填补这一空白，这是一种衡量个体政策制定者参与和使用研究能力的自我报告工具。
我们使用SPIRIT行动框架来确定相关领域，并指导用于测量每个领域的条目的开发。量表涵盖（1）使用研究的个人能力（使用研究的信心、对研究的重视程度、个人对其组织对研究重视程度的看法、支持工具和系统），（2）与研究和研究人员互动所采取的行动，以及（3）利用研究为政策提供信息（研究使用的程度和类型）。参与卫生政策制定的政策制定者样本提供数据，以检验量表的信度（内部一致性、重测）和效度（与类似概念测量的关系、与研究使用意图测量的关系、个人能力量表的内部结构）。
效度和内部一致性分析的回复率为55%（150/272人，12个机构），重测信度的回复率为54%（57/105人，9个机构）。个人能力量表显示出足够的内部一致性信度（所有四个量表的α系数>0.7）和重测信度（三个量表的组内相关系数>0.7，第四个量表为0.59）。个人能力量表的得分如预期那样与类似概念的测量结果趋同（中度相关性>0.4），验证性因素分析提供了证据，表明这些量表测量的是相关但不同的概念。这四个量表中的每个条目的关系如预期那样与源自SPIRIT行动框架的测量模型中的概念相关。关于研究参与行动和研究使用量表的信度和效度的证据不明确。
SEER的初步测试表明，这四个个人能力量表可用于政策环境中，以检查当前能力并确定能力建设领域。能力、研究参与行动和研究使用之间的关系需要进一步研究。
