is a General and Colorectal Surgeon, Division of General Surgery, Department of Surgery, St. Paul's Hospital, University of British Columbia, Vancouver, British Columbia, Canada, and Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada.
is a Rheumatologist, Director of CBME Implementation, and Clinicia Educator, Division of Rheumatology, Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada.
J Grad Med Educ. 2024 Feb;16(1):23-29. doi: 10.4300/JGME-D-23-00210.1. Epub 2024 Feb 17.
Competency-based medical education (CBME) has been implemented in many residency training programs across Canada. A key component of CBME is documentation of frequent low-stakes workplace-based assessments to track trainee progression over time. Critically, the quality of narrative feedback is imperative for trainees to accumulate a body of evidence of their progress. Suboptimal narrative feedback will challenge accurate decision-making, such as promotion to the next stage of training. To explore the quality of documented feedback provided on workplace-based assessments by examining and scoring narrative comments using a published quality scoring framework. We employed a retrospective cohort secondary analysis of existing data using a sample of 25% of entrustable professional activity (EPA) observations from trainee portfolios from 24 programs in one institution in Canada from July 2019 to June 2020. Statistical analyses explore the variance of scores between programs (Kruskal-Wallis rank sum test) and potential associations between program size, CBME launch year, and medical versus surgical specialties (Spearman's rho). Mean quality scores of 5681 narrative comments ranged from 2.0±1.2 to 3.4±1.4 out of 5 across programs. A significant and moderate difference in the quality of feedback across programs was identified (χ=321.38, <.001, ε2=0.06). Smaller programs and those with an earlier launch year performed better (<.001). No significant difference was found in quality score when comparing surgical/procedural and medical programs that transitioned to CBME in this institution (=.65). This study illustrates the complexity of examining the quality of narrative comments provided to trainees through EPA assessments.
基于能力的医学教育（CBME）已经在加拿大的许多住院医师培训项目中实施。CBME 的一个关键组成部分是记录频繁的低风险基于工作场所的评估，以跟踪学员随时间的进展。至关重要的是，叙述性反馈的质量对于学员积累他们进步的证据至关重要。次优的叙述性反馈将挑战准确的决策，例如晋升到下一阶段的培训。本研究通过使用已发表的质量评分框架检查和评分叙述性评论，探讨了基于工作场所评估提供的书面反馈的质量。我们采用了回顾性队列二次分析，使用加拿大一个机构的 24 个项目中 25%的可委托专业活动（EPA）观察值从学员档案中抽取样本，从 2019 年 7 月至 2020 年 6 月进行。统计分析探讨了项目之间评分的差异（Kruskal-Wallis 秩和检验）以及项目规模、CBME 启动年份、医学与外科专业之间的潜在关联（Spearman's rho）。5681 条叙述性评论的平均质量评分范围为 5 分制的 2.0±1.2 至 3.4±1.4 分。在不同项目之间，反馈质量存在显著的中度差异（χ=321.38，<.001，ε2=0.06）。较小的项目和启动年份较早的项目表现更好（<.001）。在比较该机构从 CBME 过渡过来的外科/程序和医学项目的质量评分时，没有发现显著差异（=.65）。本研究说明了通过 EPA 评估检查向学员提供的叙述性评论质量的复杂性。
