Pettigrew Jonathan, Graham John W, Miller-Day Michelle, Hecht Michael L, Krieger Janice L, Shin Young Ju
Department of Communication Arts and Sciences, Penn State, State College, PA, USA,
Prev Sci. 2015 Jan;16(1):90-9. doi: 10.1007/s11121-014-0459-1.
Poor implementation quality (IQ) is known to reduce program effects making it important to consider IQ for evaluation and dissemination of prevention programs. However, less is known about the ways specific implementation variables relate to outcomes. In this study, two versions of keepin' it REAL, a seventh-grade drug prevention intervention, were implemented in 78 classrooms in 25 schools in rural districts in Pennsylvania and Ohio. IQ was measured through observational coding of 276 videos. IQ variables included adherence to the curriculum, teacher engagement (attentiveness, enthusiasm, seriousness, clarity, positivity), student engagement (attention, participation), and a global rating of teacher delivery quality. Factor analysis showed that teacher engagement, student engagement, and delivery quality formed one factor, which was labeled delivery. A second factor was adherence to the curriculum. Self-report student surveys measured substance use, norms (beliefs about prevalence and acceptability of use), and efficacy (beliefs about one's ability to refuse substance offers) at two waves (pretest, immediate posttest). Mixed model regression analysis which accounted for missing data and controlled for pretest levels examined implementation quality's effects on individual level outcomes, statistically controlling for cluster level effects. Results show that when implemented well, students show positive outcomes compared to students receiving a poorly implemented program. Delivery significantly influenced substance use and norms, but not efficacy. Adherence marginally significantly predicted use and significantly predicted norms, but not efficacy. Findings underscore the importance of comprehensively measuring and accounting for IQ, particularly delivery, when evaluating prevention interventions.
众所周知，实施质量不佳会降低项目效果，因此在评估和推广预防项目时考虑实施质量非常重要。然而，对于特定实施变量与结果之间的关联方式，我们了解得较少。在本研究中，针对七年级学生的毒品预防干预项目“保持真实”（keepin' it REAL）的两个版本，在宾夕法尼亚州和俄亥俄州农村地区的25所学校的78间教室中实施。通过对276个视频进行观察编码来衡量实施质量。实施质量变量包括对课程的依从性、教师参与度（专注度、热情度、认真度、清晰度、积极性）、学生参与度（注意力、参与度）以及对教师授课质量的整体评分。因子分析表明，教师参与度、学生参与度和授课质量构成一个因子，标记为授课。第二个因子是对课程的依从性。通过学生自陈式调查在两个阶段（预测试、即时后测）测量物质使用情况、规范（对使用流行程度和可接受性的信念）以及效能感（对自己拒绝物质提供能力的信念）。混合模型回归分析考虑了缺失数据并控制了预测试水平，检验了实施质量对个体层面结果的影响，并在统计上控制了聚类层面的影响。结果表明，与接受实施不佳项目的学生相比，实施良好时学生表现出积极的结果。授课显著影响物质使用和规范，但不影响效能感。依从性对物质使用有边缘显著的预测作用，对规范有显著预测作用，但对效能感没有影响。研究结果强调了在评估预防干预措施时全面测量和考虑实施质量，特别是授课质量的重要性。
