Kavanagh Bianca E, Versace Vincent L, Beks Hannah, Dennis Carly, Binder Marley J, Hernan Andrea L, Namara Kevin Mc
Deakin University, Deakin Rural Health, School of Medicine, Warrnambool, Victoria, Australia.
Brophy Family and Youth Services, Warrnambool, Victoria, Australia.
PLoS One. 2025 Jul 30;20(7):e0322906. doi: 10.1371/journal.pone.0322906. eCollection 2025.
Community service organisations are increasingly required to report on outcomes and evaluate program delivery. While commonplace in clinical health settings, such work is novel to the community sector and can be challenging to undertake given resourcing and evaluation capacity constraints. These constraints are exacerbated for rural community service organisations that face additional resource pressures and lack organisational systems to support such work. This formative evaluation reports on an evaluation capacity building program within a rural place-based community service organisation in southwest Victoria, Australia. As part of the program, monitoring, evaluation, and learning pilots were implemented to support selected teams (N = 4 teams, N = 12 individuals) to increase confidence in outcomes measurement and reporting. Implementation strategies included training, creation of measurement frameworks, implementation of data collection tools, academic support, and emergence of evaluation champions. Evaluation data included administrative records and a staff survey. The Reach Effectiveness Adoption Implementation Maintenance framework guided evaluation. Results indicated that participants reported high awareness, beliefs, attitudes, knowledge, and skills in relation to monitoring, evaluation, and learning at the post-training/refinement period timepoint. However, several barriers limited implementation: reduced workforce capacity, prioritisation of client-driven or other work, leave or staff changes, additional responsibilities, waxing and waning engagement, wider program teams being unclear on the process and value of evaluation, and organisational barriers. These barriers led to a divergence from the planned outcome measures. Enablers to this work included physical presence to facilitate informal discussion, flexibility and openness to change, and communication to support staff. Learnings include the need to strengthen organisational evaluation culture, scaling-back the number of outcomes/data collection sources, and greater involvement with the wider staff cohort. These learnings provide a valuable foundation for other place-based community service organisations to implement evaluation capacity building strategies.
社区服务组织越来越需要报告成果并评估项目实施情况。虽然在临床健康环境中这很常见，但此类工作对于社区部门来说是新颖的，并且鉴于资源和评估能力的限制，开展起来可能具有挑战性。对于面临额外资源压力且缺乏支持此类工作的组织系统的农村社区服务组织而言，这些限制更加突出。本形成性评估报告了澳大利亚维多利亚州西南部一个基于地点的农村社区服务组织内的评估能力建设项目。作为该项目的一部分，实施了监测、评估和学习试点，以支持选定的团队（N = 4个团队，N = 12个人）增强对成果衡量和报告的信心。实施策略包括培训、创建衡量框架、实施数据收集工具、学术支持以及培养评估倡导者。评估数据包括行政记录和员工调查。采用“到达、效果、采用、实施、维持”框架指导评估。结果表明，在培训/完善期时间点，参与者报告在监测、评估和学习方面具有较高的意识、信念、态度、知识和技能。然而，若干障碍限制了实施：劳动力能力下降、优先处理客户驱动的工作或其他工作、请假或人员变动、额外职责、参与度的起伏、更广泛的项目团队对评估过程和价值不清楚以及组织障碍。这些障碍导致与计划的成果衡量产生偏差。此项工作的促成因素包括实地存在以促进非正式讨论、灵活性和对变革的开放性以及支持员工的沟通。经验教训包括需要加强组织评估文化、减少成果/数据收集来源的数量以及让更广泛的员工群体更多地参与。这些经验教训为其他基于地点的社区服务组织实施评估能力建设策略提供了宝贵的基础。
