Weiner Bryan J, Mettert Kayne D, Dorsey Caitlin N, Nolen Elspeth A, Stanick Cameo, Powell Byron J, Lewis Cara C
Department of Global Health, University of Washington, Seattle, WA, USA.
Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA.
Implement Res Pract. 2020 Aug 26;1:2633489520933896. doi: 10.1177/2633489520933896. eCollection 2020 Jan-Dec.
Systematic measure reviews can facilitate advances in implementation research and practice by locating reliable, valid, pragmatic measures; identifying promising measures needing refinement and testing; and highlighting measurement gaps. This review identifies and evaluates the psychometric and pragmatic properties of measures of readiness for implementation and its sub-constructs as delineated in the Consolidated Framework for Implementation Research: leadership engagement, available resources, and access to knowledge and information.
The systematic review methodology is described fully elsewhere. The review, which focused on measures used in mental or behavioral health, proceeded in three phases. Phase I, data collection, involved search string generation, title and abstract screening, full text review, construct assignment, and cited citation searches. Phase II, data extraction, involved coding relevant psychometric and pragmatic information. Phase III, data analysis, involved two trained specialists independently rating each measure using Psychometric and Pragmatic Evidence Rating Scales (PAPERS). Frequencies and central tendencies summarized information availability and PAPERS ratings.
Searches identified 9 measures of readiness for implementation, 24 measures of leadership engagement, 17 measures of available resources, and 6 measures of access to knowledge and information. Information about internal consistency was available for most measures. Information about other psychometric properties was often not available. Ratings for internal consistency were "adequate" or "good." Ratings for other psychometric properties were less than "adequate." Information on pragmatic properties was most often available regarding cost, language readability, and brevity. Information was less often available regarding training burden and interpretation burden. Cost and language readability generally exhibited "good" or "excellent" ratings, interpretation burden generally exhibiting "minimal" ratings, and training burden and brevity exhibiting mixed ratings across measures.
Measures of readiness for implementation and its sub-constructs used in mental health and behavioral health care are unevenly distributed, exhibit unknown or low psychometric quality, and demonstrate mixed pragmatic properties. This review identified a few promising measures, but targeted efforts are needed to systematically develop and test measures that are useful for both research and practice.
Successful implementation of effective mental health or behavioral health treatments in service delivery settings depends in part on the readiness of the service providers and administrators to implement the treatment; the engagement of organizational leaders in the implementation effort; the resources available to support implementation, such as time, money, space, and training; and the accessibility of knowledge and information among service providers about the treatment and how it works. It is important that the methods for measuring these factors are dependable, accurate, and practical; otherwise, we cannot assess their presence or strength with confidence or know whether efforts to increase their presence or strength have worked. This systematic review of published studies sought to identify and evaluate the quality of questionnaires (referred to as measures) that assess readiness for implementation, leadership engagement, available resources, and access to knowledge and information. We identified 56 measures of these factors and rated their quality in terms of how dependable, accurate, and practical they are. Our findings indicate there is much work to be done to improve the quality of available measures; we offer several recommendations for doing so.
系统的测量评价可通过找出可靠、有效且实用的测量方法，识别需要完善和测试的有前景的测量方法，以及突出测量差距，来推动实施研究和实践的进展。本评价识别并评估了实施准备度及其子结构的测量方法的心理测量学和实用特性，这些子结构在《实施研究综合框架》中有所描述：领导参与度、可用资源以及知识和信息获取情况。
系统评价方法在其他地方有详细描述。本评价聚焦于精神或行为健康领域使用的测量方法，分三个阶段进行。第一阶段，数据收集，包括生成检索词、标题和摘要筛选、全文审查、结构赋值以及被引文献检索。第二阶段，数据提取，包括对相关心理测量学和实用信息进行编码。第三阶段，数据分析，包括两名经过培训的专家使用心理测量学和实用证据评级量表（PAPERS）对每个测量方法进行独立评级。频率和集中趋势总结了信息可得性和PAPERS评级。
检索共识别出9种实施准备度测量方法、24种领导参与度测量方法、17种可用资源测量方法以及6种知识和信息获取情况测量方法多数测量方法有关于内部一致性的信息。关于其他心理测量学特性的信息通常没有。内部一致性评级为“充分”或“良好”。其他心理测量学特性的评级低于“充分”。关于实用特性的信息，成本、语言可读性和简洁性方面最常可得。关于培训负担和解释负担的信息较少可得。成本和语言可读性总体表现为“良好”或“优秀”评级，解释负担总体表现为“最小”评级，培训负担和简洁性在各测量方法中表现不一。
精神健康和行为健康护理中使用的实施准备度及其子结构的测量方法分布不均，心理测量学质量未知或较低，实用特性表现不一。本评价识别出了一些有前景的测量方法，但需要有针对性地努力，系统地开发和测试对研究和实践都有用的测量方法。
在服务提供环境中成功实施有效的精神健康或行为健康治疗，部分取决于服务提供者和管理人员实施治疗的准备程度；组织领导者对实施工作的参与度；支持实施的可用资源，如时间、资金、空间和培训；以及服务提供者获取关于治疗及其作用机制的知识和信息。重要的是，测量这些因素的方法要可靠、准确且实用；否则，我们无法自信地评估它们的存在或强度，也不知道增加它们的存在或强度的努力是否有效。这项对已发表研究的系统评价旨在识别和评估用于评估实施准备度、领导参与度、可用资源以及知识和信息获取情况的问卷（称为测量方法）的质量。我们识别出了这些因素的56种测量方法，并根据它们的可靠程度、准确程度和实用程度对其质量进行了评级。我们的研究结果表明，要提高现有测量方法的质量还有很多工作要做；我们为此提供了一些建议。
