School of Education, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
School of Social Work, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
Implement Sci. 2024 Aug 6;19(1):58. doi: 10.1186/s13012-024-01390-8.
Implementation support practitioners (ISPs) are professionals that support others to implement evidence-informed practices, programs, and policies in various service delivery settings to achieve population outcomes. Measuring the use of competencies by ISPs provides a unique opportunity to assess an understudied facet of implementation science-how knowledge, attitudes, and skills used by ISPs affects sustainable change in complicated and complex service systems. This study describes the development and validation of a measure-the Implementation Support Competencies Assessment (ISCA)-that assesses implementation support competencies, with versatile applications across service contexts.
Recently developed practice guide materials included operationalizations of core competencies for ISPs across three domains: co-creation and engagement, ongoing improvement, and sustaining change. These operationalizations, in combination with recent empirical and conceptual work, provided an initial item pool and foundation on which to advance measurement development, largely from a confirmatory perspective (as opposed to exploratory). The measure was further refined through modified cognitive interviewing with three highly experienced ISPs and pilot-testing with 39 individuals enrolled in a university-based certificate program in implementation practice. To recruit a sample for validation analyses, we leveraged a listserv of nearly 4,000 individuals who have registered for or expressed interest in various events and trainings focused on implementation practice offered by an implementation science collaborative housed within a research-intensive university in the Southeast region of the United States. Our final analytic sample included 357 participants who self-identified as ISPs.
Assessments of internal consistency reliability for each competency-specific item set yielded evidence of strong reliability. Results from confirmatory factor analyses provided evidence for the factorial and construct validity of all three domains and associated competencies in the ISCA.
The findings suggest that one's possession of high levels of competence across each of the three competency domains is strongly associated with theorized outcomes that can promote successful and sustainable implementation efforts among those who receive implementation support from an ISP. The ISCA serves as a foundational tool for workforce development to formally measure and assess improvement in the skills that are required to tailor a package of implementation strategies situated in context.
实施支持从业者（ISP）是支持他人在各种服务提供环境中实施循证实践、计划和政策以实现人群结果的专业人员。衡量 ISP 能力的使用情况为评估实施科学的一个研究不足的方面提供了独特的机会，即 ISP 使用的知识、态度和技能如何影响复杂和复杂服务系统中的可持续变化。本研究描述了一种衡量实施支持能力评估（ISCA）的开发和验证，该评估衡量实施支持能力，在服务背景下具有多种应用。
最近开发的实践指南材料包括三个领域的 ISP 核心能力的操作化：共同创造和参与、持续改进和维持变化。这些操作化，结合最近的实证和概念工作，提供了一个初始项目池和基础，主要从验证性的角度（而不是探索性的）来推进测量的发展。该措施通过与三位经验丰富的 ISP 进行修改后的认知访谈以及在 39 名参加大学实施实践证书课程的个人中进行试点测试进一步完善。为了招募验证分析的样本，我们利用了一个近 4000 人的名单，这些人已经注册或表示有兴趣参加由美国东南部一个研究密集型大学内的实施科学合作机构举办的各种实施实践相关的活动和培训。我们的最终分析样本包括 357 名自认为是 ISP 的参与者。
对每个特定能力项目集的内部一致性可靠性评估提供了强有力的可靠性证据。验证性因素分析的结果为 ISCA 中所有三个领域和相关能力的因子和结构有效性提供了证据。
这些发现表明，一个人在三个能力领域中都具有高水平的能力，这与从 ISP 获得实施支持的人成功和可持续实施努力的理论结果密切相关。ISCA 是劳动力发展的基础工具，可用于正式衡量和评估在背景中定制实施策略包所需技能的提高。
