Suzanne Dworak-Peck School of Social Work, University of Southern California, Los Angeles, CA, USA.
Department of Preventive Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.
Implement Sci. 2020 Sep 3;15(1):71. doi: 10.1186/s13012-020-01030-x.
Enhancing the sustainability of evidence-based prevention programs for mental and behavioral health requires tools for measuring both sustainability determinants and sustainment outcomes. The aim of this study was to develop the Sustainment Measurement System Scale (SMSS) and to assess its reliability and construct validity for measuring both determinants and outcomes of efforts to sustain prevention programs and initiatives.
A 42-item scale comprised of items identified from qualitative data collected from 45 representatives of 10 programs and 8 SAMHSA program officers was administered to 186 representatives of 145 programs funded by 7 SAMHSA prevention grant initiatives. Cronbach's alphas were used to determine inter-item reliability. Convergent validity was assessed by comparisons of a global measure of sustainment with current SAMHSA-funding status and continued operation in the same form. Discriminant validity was assessed by comparisons of sustainability determinants with whether or not the program had undergone adaptations.
Confirmatory factor analysis provided support for a 35-item model fit to the data. Cronbach's alpha was .84 for the sustainment outcome construct and ranged from .70 to .93 for the sustainability determinant constructs. All of the determinant constructs were significantly associated with sustainment outcome individual and global measures for the entire sample (p < 0.01 to 0.001) and for community-based programs and programs with a substance abuse focus (p < 0.05 to 0.001). Convergent validity was supported by significant associations between the global sustainment measure and current SAMHSA funding status and continued operation in the same form (p < 0.001). Four of the sustainability determinant constructs (responsive to community needs; coalitions, partnerships, and networks; organizational staff capability; and evaluation, feedback, and program outcomes) were also significantly associated with current SAMHSA funding status (p < 0.5 to 0.01). With the exception of organizational staff capability, all sustainability determinants were unrelated to program adaptation as predicted.
The SMSS demonstrated good reliability and convergent and discriminant validity in assessing likelihood of sustainment of SAMHSA funded prevention programs and initiatives. The measure demonstrates potential in identifying predictors of program sustainment and as a tool for enhancing the likelihood of successful sustainment through ongoing evaluation and feedback.
为了提高基于证据的精神和行为健康预防计划的可持续性，需要有工具来衡量可持续性决定因素和维持结果。本研究的目的是开发维持测量系统量表（SMSS），并评估其可靠性和构建有效性，以衡量维持预防计划和举措的努力的决定因素和结果。
一个由 45 名来自 10 个项目的代表和 8 名 SAMHSA 项目官员从定性数据中确定的 42 个项目组成的量表，被分发给来自 7 个 SAMHSA 预防拨款计划资助的 145 个项目的 186 名代表。采用克朗巴赫的阿尔法系数来确定项目间的可靠性。通过将维持的整体衡量标准与当前的 SAMHSA 资助状况和以相同形式继续运作进行比较，评估了聚合有效性。通过将可持续性决定因素与项目是否进行了调整进行比较，评估了区分有效性。
验证性因子分析为适用于数据的 35 项模型提供了支持。维持结果构建的克朗巴赫阿尔法系数为.84，可持续性决定因素构建的范围从.70 到.93。所有的决定因素结构都与整个样本的维持结果个体和整体测量显著相关（p < 0.01 到 0.001），以及社区为基础的项目和以药物滥用为重点的项目（p < 0.05 到 0.001）。通过全球维持措施与当前 SAMHSA 资助状况和以相同形式继续运作之间的显著关联，支持了聚合有效性（p < 0.001）。四个可持续性决定因素结构（对社区需求的响应；联盟、伙伴关系和网络；组织人员能力；以及评估、反馈和计划结果）也与当前的 SAMHSA 资助状况显著相关（p < 0.5 到 0.01）。除了组织人员能力之外，所有的可持续性决定因素都与预期的计划调整无关。
SMSS 在评估 SAMHSA 资助的预防计划和举措的维持可能性方面表现出良好的可靠性、聚合和区分有效性。该措施在确定计划维持的预测因素方面具有潜力，并可作为通过持续评估和反馈提高成功维持可能性的工具。
