Assessment of service provider competency for child and adolescent psychological treatments and psychosocial services in global mental health: evaluation of feasibility and reliability of the WeACT tool in Gaza, Palestine.

BACKGROUND

There is a scarcity of evaluated tools to assess whether non-specialist providers achieve minimum levels of competency to effectively and safely deliver psychological interventions in low- and middle-income countries. The objective of this study was to evaluate the reliability and utility of the newly developed Working with children - Assessment of Competencies Tool (WeACT) to assess service providers' competencies in Gaza, Palestine.

METHODS

The study evaluated; (1) psychometric properties of the WeACT based on observed role-plays by trainers/supervisors ( = 8); (2) sensitivity to change among service provider competencies ( = 25) using pre-and-post training WeACT scores on standardized role-plays; (3) in-service competencies among experienced service providers ( = 64) using standardized role-plays.

RESULTS

We demonstrated moderate interrater reliability [intraclass correlation coefficient, single measures, ICC = 0.68 (95% CI 0.48-0.86)] after practice, with high internal consistency ( = 0.94). WeACT assessments provided clinically relevant information on achieved levels of competencies (55% of the competencies were scored as adequate pre-training; 71% post-training; 62% in-service). Pre-post training assessment saw significant improvement in competencies ( = -3.64; < 0.001).

CONCLUSION

This study demonstrated positive results on the reliability and utility of the WeACT, with sufficient inter-rater agreement, excellent internal consistency, sensitivity to assess change, and providing insight needs for remedial training. The WeACT holds promise as a tool for monitoring quality of care when implementing evidence-based care at scale.