Mooney Christopher J, Powell Stephen Joseph, Dahl Spencer, Eiduson Carly, Reinhardt Benjamin, Stone Robert Thompson
From the Departments of Medicine (C.J.M.), and Neurology (S.J.P., B.R., R.T.S.), and Offices for Medical Education (S.D., C.E.), University of Rochester School of Medicine and Dentistry, NY.
Neurol Educ. 2022 Sep 22;1(1):e200003. doi: 10.1212/NE9.0000000000200003. eCollection 2022 Sep.
Narrative-based evaluations are increasingly used to discriminate between levels of trainee performance, yet barriers to high-quality narratives remain. Prior evidence shows mixed results regarding the effectiveness of faculty development efforts on improving narrative evaluation quality.
We used a quasi-experimental study incorporating a historical control group to examine the effectiveness of a pragmatic, multipronged, 4-year faculty development initiative on narrative evaluation quality in a neurology clerkship. We evaluated narrative evaluation quality using the narrative evaluation quality instrument (NEQI) in random samples of narrative evaluations from a historical control and intervention group. We used multilevel modeling to compare NEQI scores (and subscale scores) across groups. Informed by the theory of deliberate practice, our faculty development initiative included (1) annual grand rounds sessions focused on developing high-quality narratives and reporting evaluation metrics, (2) restructuring the clerkship assessment form to simplify and prioritize narratives, (3) recruiting key faculty to rotate on the clerkship grading committee to gain experience with and practice developing quality narratives, and (4) instituting a narrative evaluation excellence award to faculty and residents.
The faculty development initiative was associated with improvements in the quality of students' narrative evaluations. Specifically, the intervention group was a significant predictor of NEQI score, with means of 6.4 (95% CI 5.9-6.9) and 7.6 (95% CI 7.2-8.1) for the historical control and intervention groups, respectively. In addition, the intervention group was associated with significant improvement in the specificity and usefulness NEQI subscale scores, but not the performance domain subscale score.
A long-term, multipronged faculty development initiative can facilitate improvements in narrative evaluation quality. We attribute these findings to 2 factors: (1) pragmatic, solution-oriented efforts that balance focused didactics with programmatic shifts that promote deliberate practice and skill improvement and (2) departmental resources that prioritize and convey a commitment to improving trainee assessment.
基于叙事的评估越来越多地用于区分实习医生的表现水平，但高质量叙事仍存在障碍。先前的证据表明，教师发展努力对提高叙事评估质量的有效性结果不一。
我们采用了一项纳入历史对照组的准实验研究，以检验一项务实、多方面、为期4年的教师发展计划对神经科实习中叙事评估质量的有效性。我们使用叙事评估质量工具（NEQI）对历史对照组和干预组的叙事评估随机样本进行叙事评估质量评估。我们使用多层次模型比较各组的NEQI分数（及子量表分数）。基于刻意练习理论，我们的教师发展计划包括：（1）每年举办专注于高质量叙事和报告评估指标的大查房会议；（2）重新构建实习评估表以简化叙事并确定其优先级；（3）招募关键教师轮流参与实习评分委员会，以获取编写高质量叙事的经验并进行实践；（4）设立叙事评估优秀奖，奖励教师和住院医师。
教师发展计划与学生叙事评估质量的提高相关。具体而言，干预组是NEQI分数的显著预测因素，历史对照组和干预组的平均分分别为6.4（95%CI 5.9 - 6.9）和7.6（95%CI 7.2 - 8.1）。此外，干预组与NEQI子量表分数在特异性和有用性方面的显著改善相关，但与表现领域子量表分数无关。
一项长期、多方面的教师发展计划可促进叙事评估质量的提高。我们将这些发现归因于两个因素：（1）务实、以解决方案为导向的努力，在专注的教学与促进刻意练习和技能提升的计划性转变之间取得平衡；（2）部门资源优先并传达对改善实习医生评估的承诺。
