Health Research Methodology Program, McMaster University, 1280 Main St, West, CRL-209, Hamilton, ON L8S 4K1, Canada.
Health Res Policy Syst. 2011 Jun 24;9:29. doi: 10.1186/1478-4505-9-29.
Although measures of knowledge translation and exchange (KTE) effectiveness based on the theory of planned behavior (TPB) have been used among patients and providers, no measure has been developed for use among health system policymakers and stakeholders. A tool that measures the intention to use research evidence in policymaking could assist researchers in evaluating the effectiveness of KTE strategies that aim to support evidence-informed health system decision-making. Therefore, we developed a 15-item tool to measure four TPB constructs (intention, attitude, subjective norm and perceived control) and assessed its face validity through key informant interviews.
We carried out a reliability study to assess the tool's internal consistency and test-retest reliability. Our study sample consisted of 62 policymakers and stakeholders that participated in deliberative dialogues. We assessed internal consistency using Cronbach's alpha and generalizability (G) coefficients, and we assessed test-retest reliability by calculating Pearson correlation coefficients (r) and G coefficients for each construct and the tool overall.
The internal consistency of items within each construct was good with alpha ranging from 0.68 to alpha = 0.89. G-coefficients were lower for a single administration (G = 0.34 to G = 0.73) than for the average of two administrations (G = 0.79 to G = 0.89). Test-retest reliability coefficients for the constructs ranged from r = 0.26 to r = 0.77 and from G = 0.31 to G = 0.62 for a single administration, and from G = 0.47 to G = 0.86 for the average of two administrations. Test-retest reliability of the tool using G theory was moderate (G = 0.5) when we generalized across a single observation, but became strong (G = 0.9) when we averaged across both administrations.
This study provides preliminary evidence for the reliability of a tool that can be used to measure TPB constructs in relation to research use in policymaking. Our findings suggest that the tool should be administered on more than one occasion when the intervention promotes an initial 'spike' in enthusiasm for using research evidence (as it seemed to do in this case with deliberative dialogues). The findings from this study will be used to modify the tool and inform further psychometric testing following different KTE interventions.
尽管基于计划行为理论（TPB）的知识转化和交流（KTE）效果衡量措施已在患者和提供者中使用，但尚未开发出用于卫生系统政策制定者和利益相关者的衡量工具。衡量决策者在决策中使用研究证据的意图的工具可以帮助研究人员评估旨在支持循证卫生系统决策的 KTE 策略的有效性。因此，我们开发了一个 15 项工具，以衡量 TPB 的四个结构（意图、态度、主观规范和感知控制），并通过关键知情人访谈评估其表面有效性。
我们进行了可靠性研究，以评估工具的内部一致性和重测信度。我们的研究样本包括 62 名参与审议对话的政策制定者和利益相关者。我们使用 Cronbach 的 alpha 和可推广性（G）系数评估内部一致性，为每个结构和整个工具计算 Pearson 相关系数（r）和 G 系数来评估重测信度。
每个结构内的项目的内部一致性良好，alpha 值在 0.68 到 alpha = 0.89 之间。单次评估的 G 系数（G = 0.34 到 G = 0.73）低于两次评估的平均值（G = 0.79 到 G = 0.89）。结构的重测信度系数范围为 r = 0.26 到 r = 0.77，以及 G = 0.31 到 G = 0.62 用于单次评估，而两次评估的平均值为 G = 0.47 到 G = 0.86。使用 G 理论时，该工具的重测信度为中等（G = 0.5），当我们在单次观察中进行概括时，但当我们在两次评估中进行平均时，它变得很强（G = 0.9）。
这项研究为一种工具的可靠性提供了初步证据，该工具可用于衡量与决策中使用研究相关的 TPB 结构。我们的研究结果表明，当干预措施促进使用研究证据的初始“高峰”时（在审议对话中似乎就是这种情况），该工具应该进行多次评估。本研究的结果将用于修改该工具，并为不同的 KTE 干预措施提供进一步的心理测量测试。
