T. Jeyalingam is an advanced fellow in luminal therapeutic endoscopy, University of Calgary, Calgary, Alberta, Canada; ORCID: http://orcid.org/0000-0002-7254-9639 .
C.M. Walsh is a staff gastroenterologist, Division of Gastroenterology, Hepatology and Nutrition, educational researcher, SickKids Learning Institute, scientist, Child Health Evaluative Sciences, SickKids Research Institute, Hospital for Sick Children, scientist, Wilson Centre, and associate professor of paediatrics, University of Toronto, Toronto, Ontario, Canada; ORCID: http://orcid.org/0000-0003-3928-703X .
Acad Med. 2022 Jul 1;97(7):1057-1064. doi: 10.1097/ACM.0000000000004661. Epub 2022 Jun 23.
Many models of competency-based medical education (CBME) emphasize assessing entrustable professional activities (EPAs). Despite the centrality of EPAs, researchers have not compared rater entrustment decisions for the same EPA across workplace- and simulation-based assessments. This study aimed to explore rater entrustment decision making across these 2 assessment settings.
An interview-based study using a constructivist grounded theory approach was conducted. Gastroenterology faculty at the University of Toronto and the University of Calgary completed EPA assessments of trainees' endoscopic polypectomy performance in both workplace and simulation settings between November 2019 and January 2021. After each assessment, raters were interviewed to explore how and why they made entrustment decisions within and across settings. Transcribed interview data were coded iteratively using constant comparison to generate themes.
Analysis of 20 interviews with 10 raters found that participants (1) held multiple meanings of entrustment and expressed variability in how they justified their entrustment decisions and scoring, (2) held personal caveats for making entrustment decisions "comfortably" (i.e., authenticity, task-related variability, opportunity to assess trainee responses to adverse events, and the opportunity to observe multiple performances over time), (3) experienced cognitive tensions between formative and summative purposes when assessing EPAs, and (4) experienced relative freedom when using simulation to formatively assess EPAs but constraint when using only simulation-based assessments for entrustment decision making.
Participants spoke about and defined entrustment variably, which appeared to produce variability in how they judged entrustment across participants and within and across assessment settings. These rater idiosyncrasies suggest that programs implementing CBME must consider how such variability affects the aggregation of EPA assessments, especially those collected in different settings. Program leaders might also consider how to fulfill raters' criteria for comfortably making entrustment decisions by ensuring clear definitions and purposes when designing and integrating workplace- and simulation-based assessments.
许多基于能力的医学教育（CBME）模型强调评估可委托的专业活动（EPAs）。尽管 EPAs 至关重要，但研究人员尚未比较同一 EPA 在工作场所和模拟评估中的评估者委托决策。本研究旨在探索这 2 种评估设置中的评估者委托决策。
采用基于访谈的建构主义扎根理论方法进行研究。多伦多大学和卡尔加里大学的胃肠病学教师在 2019 年 11 月至 2021 年 1 月期间，分别在工作场所和模拟环境中对学员内镜息肉切除术表现进行了 EPA 评估。每次评估后，对评估者进行访谈，以探讨他们在设置内和设置间做出委托决策的方式和原因。使用常规定量比较对转录访谈数据进行迭代编码，以生成主题。
对 10 名评估者的 20 次访谈进行分析发现，参与者（1）对委托有多种含义，并表达了他们在证明委托决策和评分合理性方面的可变性，（2）对做出委托决策“放心”持有个人警告（即真实性、与任务相关的可变性、评估学员对不良事件反应的机会，以及随着时间的推移观察多个表现的机会），（3）在评估 EPA 时体验到形成性和总结性目的之间的认知紧张，（4）在使用模拟进行形成性评估 EPA 时具有相对自由，但在仅使用模拟评估进行委托决策时受到限制。
参与者对委托进行了不同的描述和定义，这似乎导致他们在参与者之间以及在设置内和设置间对委托进行判断时产生了可变性。这些评估者的特质表明，实施 CBME 的计划必须考虑这种可变性如何影响 EPA 评估的聚合，尤其是那些在不同设置中收集的评估。计划负责人还可以考虑通过在设计和整合工作场所和模拟评估时确保明确的定义和目的，来满足评估者做出委托决策的条件。
