Applying RE-AIM to evaluations of Veterans Health Administration Enterprise-Wide Initiatives: lessons learned.

INTRODUCTION

The United States Veterans Health Administration (VHA) Office of Rural Health funds Enterprise-Wide Initiatives (system-wide initiatives) to spread promising practices to rural Veterans. The Office requires that evaluations of Enterprise-Wide Initiatives use the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. This presents a unique opportunity to understand the experience of using RE-AIM across a series of evaluations. The authors conducted a study to document the benefits and pitfalls of using RE-AIM, capture the variety of ways that the team captured the elements of RE-AIM, and develop recommendations for the future use of RE-AIM in evaluation.

MATERIALS AND METHODS

The authors first conducted a document review to capture pre-existing information about how RE-AIM was used. They subsequently facilitated two focus groups to gather more detailed information from team members who had used RE-AIM. Finally, they used member-checking throughout the writing process to ensure accurate data representation and interpretation and to gather additional feedback.

RESULTS

Four themes emerged from the document review, focus groups, and member checking. RE-AIM: provides parameters and controls the evaluation scope, "buckets" are logical, plays well with other frameworks, and can foster collaboration or silo within a team. Challenges and attributes for each RE-AIM dimension were also described.

DISCUSSION

Overall, participants reported both strengths and challenges to using RE-AIM as an evaluation framework. The overarching theme around the challenges with RE-AIM dimensions was the importance of context. Many of these benefits and challenges of using RE-AIM may not be unique to RE-AIM and would likely occur when using any prescribed framework. The participants reported on the RE-AIM domains in a variety of ways in their evaluation reports and were not always able capture data as originally planned. Recommendations included: start with an evaluation framework (or frameworks) and revisit it throughout the evaluation, consider applying RE-AIM PRISM (Practical Robust Implementation Framework) to gain a broader perspective, and intentionally integrate quantitative and qualitative team members, regardless of the framework used.