Lewis Cara C, Weiner Bryan J, Stanick Cameo, Fischer Sarah M
Department of Psychological and Brain Sciences, Indiana University, 1101 E. 10th St., Bloomington, IN, 47405, USA.
Department of Psychiatry and Behavioral Sciences, School of Medicine, Harborview Medical Center, University of Washington, 325 9th Ave, Box 359911, Seattle, WA, 98104, USA.
Implement Sci. 2015 Jul 22;10:102. doi: 10.1186/s13012-015-0287-0.
Significant gaps related to measurement issues are among the most critical barriers to advancing implementation science. Three issues motivated the study aims: (a) the lack of stakeholder involvement in defining pragmatic measure qualities; (b) the dearth of measures, particularly for implementation outcomes; and (c) unknown psychometric and pragmatic strength of existing measures. Aim 1: Establish a stakeholder-driven operationalization of pragmatic measures and develop reliable, valid rating criteria for assessing the construct. Aim 2: Develop reliable, valid, and pragmatic measures of three critical implementation outcomes, acceptability, appropriateness, and feasibility. Aim 3: Identify Consolidated Framework for Implementation Research and Implementation Outcome Framework-linked measures that demonstrate both psychometric and pragmatic strength.
METHODS/DESIGN: For Aim 1, we will conduct (a) interviews with stakeholder panelists (N = 7) and complete a literature review to populate pragmatic measure construct criteria, (b) Q-sort activities (N = 20) to clarify the internal structure of the definition, (c) Delphi activities (N = 20) to achieve consensus on the dimension priorities, (d) test-retest and inter-rater reliability assessments of the emergent rating system, and (e) known-groups validity testing of the top three prioritized pragmatic criteria. For Aim 2, our systematic development process involves domain delineation, item generation, substantive validity assessment, structural validity assessment, reliability assessment, and predictive validity assessment. We will also assess discriminant validity, known-groups validity, structural invariance, sensitivity to change, and other pragmatic features. For Aim 3, we will refine our established evidence-based assessment (EBA) criteria, extract the relevant data from the literature, rate each measure using the EBA criteria, and summarize the data.
The study outputs of each aim are expected to have a positive impact as they will establish and guide a comprehensive measurement-focused research agenda for implementation science and provide empirically supported measures, tools, and methods for accomplishing this work.
与测量问题相关的重大差距是推进实施科学的最关键障碍之一。有三个问题推动了本研究的目标：（a）利益相关者在定义实用测量质量方面缺乏参与；（b）测量方法匮乏，尤其是针对实施结果的测量方法；（c）现有测量方法的心理测量学和实用性强度未知。目标1：建立由利益相关者驱动的实用测量方法的操作化，并制定可靠、有效的评级标准以评估该结构。目标2：开发针对三个关键实施结果——可接受性、适宜性和可行性的可靠、有效且实用的测量方法。目标3：识别与实施研究综合框架和实施结果框架相关联的测量方法，这些方法应兼具心理测量学和实用性强度。
方法/设计：对于目标1，我们将进行以下操作：（a）与利益相关者小组成员（N = 7）进行访谈，并完成文献综述以确定实用测量结构标准；（b）进行Q分类活动（N = 20）以阐明定义的内部结构；（c）进行德尔菲活动（N = 20）以就维度优先级达成共识；（d）对新出现的评级系统进行重测和评分者间信度评估；（e）对排名前三的实用标准进行已知群体效度测试。对于目标2，我们的系统开发过程包括领域界定、项目生成、实质效度评估、结构效度评估、信度评估和预测效度评估。我们还将评估区分效度、已知群体效度、结构不变性、对变化的敏感性以及其他实用特征。对于目标3，我们将完善已建立的循证评估（EBA）标准，从文献中提取相关数据，使用EBA标准对每个测量方法进行评分，并汇总数据。
预计每个目标的研究成果都将产生积极影响，因为它们将为实施科学建立并指导一个以测量为重点的全面研究议程，并为完成这项工作提供实证支持的测量方法、工具和手段。
