Department of Global Health, University of Washington, 1510 San Juan Road, Box 357965, Seattle, WA, 98195, USA.
Kaiser Permanente Washington Health Research Institute, MacColl Center for Health Care Innovation, 1730 Minor Avenue, Suite 1600, Seattle, WA, 98101, USA.
Implement Sci. 2017 Aug 29;12(1):108. doi: 10.1186/s13012-017-0635-3.
Implementation outcome measures are essential for monitoring and evaluating the success of implementation efforts. Yet, currently available measures lack conceptual clarity and have largely unknown reliability and validity. This study developed and psychometrically assessed three new measures: the Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM).
Thirty-six implementation scientists and 27 mental health professionals assigned 31 items to the constructs and rated their confidence in their assignments. The Wilcoxon one-sample signed rank test was used to assess substantive and discriminant content validity. Exploratory and confirmatory factor analysis (EFA and CFA) and Cronbach alphas were used to assess the validity of the conceptual model. Three hundred twenty-six mental health counselors read one of six randomly assigned vignettes depicting a therapist contemplating adopting an evidence-based practice (EBP). Participants used 15 items to rate the therapist's perceptions of the acceptability, appropriateness, and feasibility of adopting the EBP. CFA and Cronbach alphas were used to refine the scales, assess structural validity, and assess reliability. Analysis of variance (ANOVA) was used to assess known-groups validity. Finally, half of the counselors were randomly assigned to receive the same vignette and the other half the opposite vignette; and all were asked to re-rate acceptability, appropriateness, and feasibility. Pearson correlation coefficients were used to assess test-retest reliability and linear regression to assess sensitivity to change.
All but five items exhibited substantive and discriminant content validity. A trimmed CFA with five items per construct exhibited acceptable model fit (CFI = 0.98, RMSEA = 0.08) and high factor loadings (0.79 to 0.94). The alphas for 5-item scales were between 0.87 and 0.89. Scale refinement based on measure-specific CFAs and Cronbach alphas using vignette data produced 4-item scales (α's from 0.85 to 0.91). A three-factor CFA exhibited acceptable fit (CFI = 0.96, RMSEA = 0.08) and high factor loadings (0.75 to 0.89), indicating structural validity. ANOVA showed significant main effects, indicating known-groups validity. Test-retest reliability coefficients ranged from 0.73 to 0.88. Regression analysis indicated each measure was sensitive to change in both directions.
The AIM, IAM, and FIM demonstrate promising psychometric properties. Predictive validity assessment is planned.
实施结果衡量标准对于监测和评估实施工作的成功至关重要。然而，目前可用的衡量标准缺乏概念清晰度，并且其可靠性和有效性在很大程度上未知。本研究开发并心理测量评估了三个新的衡量标准：干预措施可接受性衡量标准（AIM）、干预措施适当性衡量标准（IAM）和干预措施可行性衡量标准（FIM）。
36 名实施科学家和 27 名心理健康专业人员将 31 个项目分配给各个结构，并对其分配的置信度进行评分。采用 Wilcoxon 单样本符号秩检验来评估实质内容和区分内容的有效性。采用探索性和验证性因素分析（EFA 和 CFA）以及克朗巴赫阿尔法系数来评估概念模型的有效性。326 名心理健康顾问阅读了六个随机分配的案例描述之一，描述了一位治疗师考虑采用循证实践（EBP）。参与者使用 15 个项目来评估治疗师对采用 EBP 的可接受性、适当性和可行性的看法。采用 CFA 和克朗巴赫阿尔法系数来改进量表、评估结构有效性和评估可靠性。采用方差分析（ANOVA）来评估已知组有效性。最后，一半的顾问被随机分配阅读相同的案例，另一半阅读相反的案例；并要求他们重新评估可接受性、适当性和可行性。采用 Pearson 相关系数来评估测试-重测信度，采用线性回归来评估对变化的敏感性。
除了五项之外，所有项目都表现出实质性和区分性的内容有效性。具有五个项目/结构的修剪 CFA 表现出可接受的模型拟合度（CFI=0.98，RMSEA=0.08）和较高的因子负荷（0.79 至 0.94）。5 项量表的阿尔法值在 0.87 至 0.89 之间。基于案例特定 CFA 和克朗巴赫阿尔法的量表细化使用案例数据产生了 4 项量表（α 值在 0.85 至 0.91 之间）。具有三个因素的 CFA 表现出可接受的拟合度（CFI=0.96，RMSEA=0.08）和较高的因子负荷（0.75 至 0.89），表明具有结构有效性。方差分析显示出显著的主效应，表明具有已知组有效性。测试-重测信度系数范围在 0.73 至 0.88 之间。回归分析表明，每个衡量标准在两个方向上都对变化敏感。
AIM、IAM 和 FIM 表现出有希望的心理测量特性。计划进行预测有效性评估。
Zotero 插件