BACKGROUND

Coaching programs in graduate medical education have the potential to impact trainee development across multiple core competencies but require rigorous program evaluation to ensure effectiveness. We sought to qualitatively evaluate the implementation of a multi-departmental, faculty-led communication coaching program using a logic model framework.

METHODS

Study participants were selected from four key stakeholder groups: resident coachees, faculty coaches, medical education leaders, and programmatic sponsors. 30-45 min semi-structured interviews were conducted via Zoom, transcribed, and de-identified for the analysis. Interviews captured stakeholders' perspectives on physicians' communication training needs, stakeholders perceived and actual roles, stakeholders' involvement in the program, factors influencing the implementation process, and strategies for programmatic improvement, sustainment, and spread. The Consolidated Framework of Implementation Research (CFIR) guided the codebook development and data analysis. A combined inductive/deductive approach was used to develop a 20-item codebook, followed by a team-based thematic analysis. A strong intercoder agreement (Cohen's kappa coefficient κ = 0.83) ensured coding consistency. The emerging themes were then mapped onto four domains of a logic model: Context, Inputs and Outputs, Outcomes, and Evaluation.

RESULTS

35 interviews were conducted between November 2021 and April 2022 with representation from all stakeholder groups, including 10 resident coachees (who received coaching), 10 faculty coaches (who served as coaches and underwent coaching-specific faculty development), 9 medical education leaders (who designed and implemented program), and programmatic sponsors (who provided financial support). We mapped 8 emergent themes onto the critical domains of a logic model for program evaluation. For the domain of Context, themes included (1) gap in communication education and (2) patient-centeredness. For the domain of Inputs/Outputs, themes included (1) investment in the program and (2) perceived program value. For the domain of Outcomes, themes included (1) learning-focused outcomes and (2) patient-related outcomes. For the domain of Evaluation, themes included (1) defining success and (2) challenges with evaluation.

CONCLUSIONS

Mapping CFIR-informed themes onto a logic model for program evaluation presents a novel strategy for integrating program implementation and evaluation, both of which are essential to effective educational programming. These findings can be used to guide future programmatic modifications to better meet the needs of key stakeholders.