Department of Internal Medicine, School of Medicine and George E. Whalen VA Hospital, University of Utah, Salt Lake City, UT, USA.
Department of Internal Medicine, School of Medicine, University of Utah, Salt Lake City, UT, USA.
J Gen Intern Med. 2019 Jun;34(6):929-935. doi: 10.1007/s11606-019-04946-3.
Feedback is a critical element of graduate medical education. Narrative comments on evaluation forms are a source of feedback for residents. As a shared mental model for performance, milestone-based evaluations may impact narrative comments and resident perception of feedback.
To determine if milestone-based evaluations impacted the quality of faculty members' narrative comments on evaluations and, as an extension, residents' perception of feedback.
Concurrent mixed methods study, including qualitative analysis of narrative comments and survey of resident perception of feedback.
Seventy internal medicine residents and their faculty evaluators at the University of Utah.
Faculty narrative comments from 248 evaluations pre- and post-milestone implementation were analyzed for quality and Accreditation Council for Graduate Medical Education competency by area of strength and area for improvement. Seventy residents were surveyed regarding quality of feedback pre- and post-milestone implementation.
Qualitative analysis of narrative comments revealed nearly all evaluations pre- and post-milestone implementation included comments about areas of strength but were frequently vague and not related to competencies. Few evaluations included narrative comments on areas for improvement, but these were of higher quality compared to areas of strength (p < 0.001). Overall resident perception of quality of narrative comments was low and did not change following milestone implementation (p = 0.562) for the 86% of residents (N = 60/70) who completed the pre- and post-surveys.
The quality of narrative comments was poor, and there was no evidence of improved quality following introduction of milestone-based evaluations. Comments on areas for improvement were of higher quality than areas of strength, suggesting an area for targeted intervention. Residents' perception of feedback quality did not change following implementation of milestone-based evaluations, suggesting that in the post-milestone era, internal medicine educators need to utilize additional interventions to improve quality of feedback.
反馈是研究生医学教育的关键要素。评估表上的叙述性评论是住院医师获得反馈的一种来源。基于里程碑的评估可能会影响叙述性评论和住院医师对反馈的感知，因为它是一种共同的绩效心理模型。
确定基于里程碑的评估是否会影响教师对评估的叙述性评论的质量，以及作为扩展，住院医师对反馈的感知。
同时进行的混合方法研究，包括对叙述性评论的定性分析和对住院医师反馈感知的调查。
犹他大学的 70 名内科住院医师及其教师评估者。
对实施里程碑前后 248 份评估的教师叙述性评论进行质量分析，并根据优势领域和改进领域进行评估委员会对研究生医学教育的胜任力分析。对 70 名住院医师进行了里程碑实施前后反馈质量的调查。
叙述性评论的定性分析显示，几乎所有实施里程碑前后的评估都包含了关于优势领域的评论，但这些评论通常很模糊，与能力无关。很少有评估包括对改进领域的叙述性评论，但与优势领域相比，这些评论的质量更高（p<0.001）。总体而言，住院医师对叙述性评论质量的感知较低，且在实施里程碑后并未改变（p=0.562），70 名住院医师中有 86%（N=60/70）完成了前后调查。
叙述性评论的质量较差，且在引入基于里程碑的评估后，没有证据表明质量有所提高。关于改进领域的评论质量高于优势领域，这表明需要有针对性地进行干预。在实施基于里程碑的评估后，住院医师对反馈质量的感知没有改变，这表明在实施里程碑后时代，内科教育工作者需要利用其他干预措施来提高反馈质量。
