Oakville Trafalgar Memorial Hospital, Oakville, Ontario, Canada.
Critical Care Medicine Residency Program, University of Toronto, Toronto, Ontario, Canada.
Perspect Med Educ. 2023 Oct 20;12(1):427-437. doi: 10.5334/pme.1020. eCollection 2023.
Receiving feedback from different types of assessors (e.g., senior residents, staff supervisors) may impact trainees' perceptions of the quantity and quality of data during entrustable professional activity (EPA) assessments. We evaluated the quality of EPA feedback provided by different assessors (senior residents, chief medical residents/subspecialty residents, and staff) and explored residents' judgements of the value of this feedback.
From a database of 2228 EPAs, we calculated the frequency of contribution from three assessor groups. We appraised the quality of 60 procedure-related EPAs completed between July 2019 and March 2020 using a modified Completed Clinical Evaluation Report Rating (CCERR) tool. Next, we asked 15 internal medicine residents to sort randomly selected EPAs according to their judgements of value, as an elicitation exercise before a semi-structured interview. Interviews explored participants' perceptions of quality of written feedback and helpful assessors.
Residents completed over 60% of EPA assessments. We found no difference in modified-CCERR scores between the three groups. When judging EPA feedback value, residents described a process of weighted deliberation, considering perceived assessor characteristics (e.g., credibility, experience with EPA system), actionable written comments, and their own self-assessment.
Like other recent studies, we found that residents contributed most to procedure-related EPA assessments. To the established list of factors influencing residents' judgements of feedback value, we add assessors' adherence to, and their shared experiences of being assessed within, EPA assessment systems. We focus on the implications for how assessors and leaders can build credibility in themselves and in the practices of EPA assessments.
从不同类型的评估者（例如，高级住院医师、主管监督者）那里获得反馈可能会影响受训者对委托专业活动（EPA）评估中数据数量和质量的感知。我们评估了不同评估者（高级住院医师、主治住院医师/专科住院医师和主管）提供的 EPA 反馈质量，并探讨了住院医师对该反馈价值的判断。
从 2228 项 EPA 中，我们计算了三组评估者的贡献频率。我们使用改良的完成临床评估报告评分（CCERR）工具评估了 2019 年 7 月至 2020 年 3 月期间完成的 60 项与程序相关的 EPA 的质量。接下来，我们让 15 名内科住院医师根据他们对价值的判断对随机选择的 EPA 进行排序，这是在半结构化访谈之前的一个启发练习。访谈探讨了参与者对书面反馈质量和有帮助的评估者的看法。
住院医师完成了超过 60%的 EPA 评估。我们没有发现三组之间改良 CCERR 评分的差异。在判断 EPA 反馈价值时，住院医师描述了一个权衡审议的过程，考虑了感知评估者的特征（例如，可信度、EPA 系统经验）、可操作的书面意见以及他们自己的自我评估。
像其他最近的研究一样，我们发现住院医师主要参与了与程序相关的 EPA 评估。在影响住院医师对反馈价值判断的因素清单上，我们增加了评估者对 EPA 评估系统的遵守情况以及他们在该系统中被评估的共同经验。我们关注的是评估者和领导者如何在自己和 EPA 评估实践中建立可信度的影响。
