Crawford Kelsey Anne, Wood Timothy J, Lalonde Karl-André, Dudek Nancy
University of Ottawa.
MedEdPublish (2016). 2019 Jan 22;8:18. doi: 10.15694/mep.2019.000018.1. eCollection 2019.
This article was migrated. The article was marked as recommended. With the advent of competency-based medical education there is an emphasis on formative workplace based assessment. The quality of these assessments is a concern for medical educators and their trainees. Faculty development (FD) strategies to improve assessment quality have resulted in some success. However, few faculty participate, and those who do are likely more motivated to improve, making it difficult to demonstrate a conclusive benefit. To address these weaknesses, we designed a FD initiative to improve the quality of completed in-training evaluation reports (ITERs). All faculty within a division participated. We hypothesized that clinical supervisors would improve their ITER quality based on feedback, regardless of their own motivation to do so, with a simple, point-in-time intervention. In this three-phase study, two independent raters used the Completed Clinical Evaluation Report Rating (CCERR) to assess the quality of ITERs completed by all faculty in the Division of Orthopedic Surgery at the University of Ottawa. In phase one, ITERs from the previous nine months were evaluated. In phase two, the participants were aware that their ITERs were being evaluated, but they did not receive feedback. In phase three, participants received regular feedback on their performance in the form of their mean CCERR scores. Mean CCERR scores from the different phases of the study were compared. CCERR scores were similar for all three phases (one: 17.56 ± 1.02, two: 17.65 ± 0.96, three: 17.54 ± 0.75, p=0.98). There was no evidence in our study that participants' improved their ITER quality despite being aware that they were being evaluated and/or receiving feedback. Potentially, this was related to a lack of motivation. Alternatively, the intensity and/or frequency of the feedback may have been inadequate to create change. These results raise concerns that some faculty development may not necessarily be better than none.
本文已迁移。该文章被标记为推荐文章。随着基于胜任力的医学教育的出现，人们开始重视基于工作场所的形成性评估。这些评估的质量是医学教育工作者及其学员关注的问题。提高评估质量的教师发展（FD）策略已取得了一些成功。然而，很少有教师参与，而且参与的教师可能更有动力去改进，这使得难以证明有确凿的益处。为了解决这些弱点，我们设计了一项教师发展倡议，以提高已完成的培训期间评估报告（ITER）的质量。一个科室的所有教师都参与了。我们假设临床督导员会根据反馈提高他们的ITER质量，无论他们自己这样做的动机如何，只需进行一次简单的、时间点明确的干预。在这项三阶段研究中，两名独立评分者使用已完成临床评估报告评分（CCERR）来评估渥太华大学骨科手术科室所有教师完成的ITER的质量。在第一阶段，对前九个月的ITER进行评估。在第二阶段，参与者知道他们的ITER正在被评估，但他们没有收到反馈。在第三阶段，参与者以其平均CCERR分数的形式收到关于他们表现的定期反馈。比较了研究不同阶段的平均CCERR分数。研究的所有三个阶段的CCERR分数相似（第一阶段：17.56±1.02，第二阶段：17.65±0.96，第三阶段：17.54±0.75，p = 0.98）。我们的研究没有证据表明参与者在知道自己正在被评估和/或收到反馈的情况下提高了他们的ITER质量。这可能与缺乏动力有关。或者，反馈的强度和/或频率可能不足以促成改变。这些结果引发了人们对某些教师发展可能不一定比不进行教师发展更好的担忧。
