C. Carraccio was vice president for competency-based medical education, American Board of Pediatrics, Chapel Hill, North Carolina, at the time of this study.
A. Martini is clinical research coordinator, Division of Emergency Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio.
Acad Med. 2021 Sep 1;96(9):1332-1336. doi: 10.1097/ACM.0000000000004075.
Competency-based assessment, using entrustable professional activities (EPAs), is rapidly being implemented worldwide without sufficient agreement on the essential elements of EPA-based assessment. The rapidity of implementation has left little time to understand what works in what circumstances and why or why not. The result is the attempted execution of a complex service intervention without a shared mental model for features needed to remain true to implementing an EPA assessment framework as intended. The purpose of this study was to identify the essential core components necessary to maintain integrity in the implementation of this intended intervention.
A formal consensus-building technique, the Delphi process, was used to identify core components for implementing an EPA-based assessment framework. Twelve EPA experts from the United States, Canada, and the Netherlands participated in this process in February and March 2020. In each Delphi round, participants rated possible core components on a scale from 1 to 6, with 1 reflecting the worst fit and 6 the best fit for EPA-based assessment implementation. Predetermined automatic inclusion and exclusion criteria for candidate core components were set at ≥ 80% of participants assigning a value of 5 or 6 and ≥ 80% assigning a value of 1 or 2, respectively.
After 3 rounds, participants prioritized 10 of 19 candidate core components for inclusion: performance prediction, shared local mental model, workplace assessment, high-stakes entrustment decisions, outcomes based, value of the collective, informed clinical competency committee members, construct alignment, qualitative data, and entrustment decision consequences. The study closed after 3 rounds on the basis of the rankings and comments.
Using the core components identified in this study advances efforts to implement an EPA assessment framework intervention as intended, which mitigates the likelihood of making an incorrect judgment that the intervention demonstrates negative results.
基于可委托专业活动（EPAs）的能力评估正在全球范围内迅速实施，但对于基于 EPA 的评估的基本要素尚未达成充分共识。实施的迅速性使得几乎没有时间去了解在什么情况下什么有效，以及为什么有效或无效。其结果是，在缺乏对实施 EPA 评估框架所需特征的共同心理模型的情况下，尝试执行一项复杂的服务干预措施，而这一框架原本是意图保持评估的真实性。本研究的目的是确定在实施这一预期干预措施时保持完整性所需的基本核心要素。
采用正式的共识建立技术——德尔菲法，确定实施基于 EPA 的评估框架的核心要素。来自美国、加拿大和荷兰的 12 名 EPA 专家于 2020 年 2 月和 3 月参与了这一过程。在每一轮德尔菲法中，参与者根据 1 到 6 的等级对可能的核心要素进行评分，1 表示最不适合，6 表示最适合基于 EPA 的评估实施。对于候选核心要素，预先设定了自动纳入和排除标准，即≥80%的参与者给予 5 或 6 的分值，且≥80%的参与者给予 1 或 2 的分值。
经过 3 轮，参与者优先考虑纳入 19 个候选核心要素中的 10 个：绩效预测、共享的局部心理模型、工作场所评估、高风险委托决策、基于结果、集体价值、知情的临床能力委员会成员、结构一致性、定性数据和委托决策结果。该研究在基于排名和评论的基础上，在 3 轮后结束。
使用本研究中确定的核心要素可以推进实施 EPA 评估框架干预措施的工作，从而降低对干预措施产生负面结果的错误判断的可能性。
