Brian Riley, Rodriguez Natalie, Zhou Connie J, Casey Megan, Mora Rosa V, Miclau Katherine, Kwok Vivian, Feldman Liane S, Alseidi Adnan
Department of Surgery, University of California San Francisco, San Francisco, CA, USA.
School of Medicine, University of California San Francisco, San Francisco, CA, USA.
Surg Open Sci. 2024 Feb 24;18:93-97. doi: 10.1016/j.sopen.2024.02.008. eCollection 2024 Mar.
Entrustable Professional Activities (EPAs) allow for the assessment of specific, observable, essential tasks in medical education. Since being developed in non-surgical fields, EPA assessments have been implemented in surgery to explore intraoperative entrustment. However, assessment burden is a significant problem for faculty, and it is unknown whether EPA assessments enable formative technical feedback. EPAs' formative utility could inform how surgical programs facilitate technical feedback for trainees. We aimed to assess the extent to which narrative comments provided through the Fellowship Council (FC) EPA assessments contained technical feedback.
The FC previously collected EPA assessments for subspecialty surgical fellows from September 2020 to October 2022. Two raters reviewed assessments' narrative comments for inclusion of each skill area that makes up part of the Objective Structured Assessment of Technical Skills (OSATS). A third rater reconciled discrepant ratings.
During the study period, there were 3302 completed EPA assessments, including 1191 fellow self-assessments, 1124 faculty assessments, and 987 assessments without an identified assessor role. We found that assessments' narrative comments related to a median of two of the seven OSATS areas (IQR:1-2). There were no comments relevant to any of the seven OSATS areas in 16.0 % of all assessments.
In this review of narrative comments for EPA assessments from the FC, we found that limited technical feedback of the kind included in the OSATS was provided in many assessments. These results suggest benefit to adjusting the EPA form, enhancing faculty development, or continuing additional types of targeted technical assessment intraoperatively.
This analysis of narrative comments from fellowship EPA assessments showed that many assessments included limited technical feedback. To allow for continued technical feedback for fellows, these results highlight the need for further refinements of the EPA assessment form, additional faculty development, or ongoing use of other types of technical assessment.
可托付专业活动（EPA）有助于评估医学教育中特定的、可观察到的关键任务。自非手术领域开发以来，EPA评估已在外科手术中实施，以探索术中托付情况。然而，评估负担对教员来说是一个重大问题，而且尚不清楚EPA评估是否能提供形成性的技术反馈。EPA的形成性效用可为外科项目如何促进对学员的技术反馈提供参考。我们旨在评估通过 fellowship Council（FC）的EPA评估提供的叙述性评论包含技术反馈的程度。
FC此前在2020年9月至2022年10月期间收集了亚专科外科住院医师的EPA评估。两名评分者审查评估的叙述性评论，以确定其是否包含构成客观结构化技术技能评估（OSATS）一部分的每个技能领域。第三名评分者协调存在差异的评分。
在研究期间，共完成了3302项EPA评估，包括1191项住院医师自我评估、1124项教员评估以及987项未明确评估者角色的评估。我们发现，评估的叙述性评论中与七个OSATS领域中的两个领域相关的中位数为2个（四分位间距：1 - 2）。在所有评估中，16.0%的评估没有与七个OSATS领域中的任何一个相关的评论。
在对FC的EPA评估叙述性评论的此次审查中，我们发现许多评估中提供的OSATS类型的技术反馈有限。这些结果表明，调整EPA表格、加强教员培训或在术中继续进行其他类型的针对性技术评估可能有益。
对住院医师EPA评估叙述性评论的分析表明，许多评估包含的技术反馈有限。为了能继续为住院医师提供技术反馈，这些结果凸显了进一步完善EPA评估表格、加强教员培训或持续使用其他类型技术评估的必要性。
