School of Clinical Sciences at Monash Health, Monash University, Clayton, Australia.
Department of Anaesthesia and Perioperative Medicine, Monash Health, Clayton, Australia.
Anaesth Intensive Care. 2024 Jul;52(4):241-249. doi: 10.1177/0310057X241234676. Epub 2024 Apr 22.
The role of self-assessment in workplace-based assessment remains contested. However, anaesthesia trainees need to learn to judge the quality of their own work. Entrustment scales have facilitated a shared understanding of performance standards among supervisors by aligning assessment ratings with everyday clinical supervisory decisions. We hypothesised that if the entrustment scale similarly helped trainees in their self-assessment, there would be substantial agreement between supervisor and trainee ratings. We collected separate mini-clinical evaluation exercises forms from 113 anaesthesia trainee-supervisor pairs from three hospitals in Australia and New Zealand. We calculated the agreement between trainee and supervisor ratings using Pearson and intraclass correlation coefficients. We also tested for associations with demographic variables and examined narrative comments for factors influencing rating. We found ratings agreed in 32% of cases, with 66% of trainee ratings within one point of the supervisor rating on a nine-point scale. The correlation between trainee and supervisor ratings was 0.71, and the degree of agreement measured by the intraclass correlation coefficient was 0.67. With higher supervisor ratings, trainee ratings better correlated with supervisor ratings. We found no strong association with demographic variables. Possible explanations of divergent ratings included one party being unaware of a vital aspect of the performance and different interpretations of the prospective nature of the scale. The substantial concordance between trainee and supervisor ratings supports the contention that the entrustment scale helped produce a shared understanding of the desired performance standard. Discussion between trainees and supervisors on the reasoning underlying their respective judgements would provide further opportunities to enhance this shared understanding.
自我评估在基于工作场所的评估中的作用仍存在争议。然而，麻醉学受训者需要学会判断自己工作的质量。委托量表通过将评估等级与日常临床监督决策保持一致，促进了主管之间对绩效标准的共识。我们假设，如果委托量表同样有助于受训者进行自我评估，那么主管和受训者的评分之间会有很大的一致性。我们从澳大利亚和新西兰的三家医院收集了 113 对麻醉学受训者-主管的独立迷你临床评估练习表。我们使用 Pearson 和组内相关系数来计算受训者和主管评分之间的一致性。我们还测试了与人口统计学变量的关联，并检查了影响评分的叙述性评论。我们发现，在 32%的情况下评分是一致的，在 9 分制中，有 66%的受训者评分与主管评分相差一分。受训者和主管评分之间的相关性为 0.71，组内相关系数衡量的一致性程度为 0.67。随着主管评分的提高，受训者的评分与主管评分的相关性越好。我们没有发现与人口统计学变量有很强的关联。评分差异的可能解释包括一方不知道绩效的一个重要方面，以及对量表前瞻性的不同解释。受训者和主管评分之间的高度一致性支持了这样一种观点，即委托量表有助于对期望的绩效标准达成共识。受训者和主管之间关于其各自判断背后的推理进行讨论，将为进一步增强这种共识提供机会。
