Fleming Melinda, Vautour Danika, McMullen Michael, Cofie Nicholas, Dalgarno Nancy, Phelan Rachel, Mizubuti Glenio B
Department of Anesthesiology and Perioperative Medicine, Kingston Health Sciences Centre.
Faculty of Health Sciences, Queens University, Ontario, Canada.
Can Med Educ J. 2021 Sep 14;12(4):17-26. doi: 10.36834/cmej.70697. eCollection 2021 Sep.
Residents' accurate self-assessment and clinical judgment are essential for optimizing their clinical skills development. Evidence from the medical literature suggests that residents generally do poorly at self-assessing their performance, often due to factors relating to learners' personal backgrounds, cultures, the specific contexts of the learning environment and rater bias or inaccuracies. We evaluated the accuracy of anesthesiology residents' self-assessed Global Entrustment scores and determined whether differences between faculty and resident scores varied by resident seniority, faculty leniency, and/or year of assessment.
We employed variance components modeling techniques and analyzed 329 pairs of faculty and self-assessed entrustment scores among 43 faculty assessors and 15 residents. Using faculty scores as the gold standard, we compared faculty scores with residents' scores (x-x), and determined residents' accuracy, including over- and under-confidence.
The results indicate that residents were respectively over- and under-confident in 10.9% and 54.4% of the assessments but more consistent in their individual self-assessments ( = 0.70) than faculty assessors. Faculty scores were significantly higher ( = 0.396; z = 4.39; < 0.001) than residents' self-assessed scores. Being a lenient/dovish ( = 0.121, z = 3.16, < 0.01) and a neutral ( = 0.137, z = 3.57, < 0.001) faculty assessor predicted a higher likelihood of resident under-confidence. Senior residents were significantly less likely to be under-confident compared to junior residents ( = -0.182, z =-2.45, < 0.05). The accuracy of self-assessments did not significantly vary during the two years of the study period.
The majority of residents' self-assessments were inaccurate. Our findings may help identify the sources of such inaccuracies.
住院医师准确的自我评估和临床判断对于优化其临床技能发展至关重要。医学文献中的证据表明，住院医师通常在自我评估表现方面做得很差，这往往是由于与学习者个人背景、文化、学习环境的具体情况以及评估者偏差或不准确等因素有关。我们评估了麻醉科住院医师自我评估的整体委托分数的准确性，并确定了教员和住院医师分数之间的差异是否因住院医师年资、教员宽松程度和/或评估年份而有所不同。
我们采用方差成分建模技术，分析了43名教员评估者和15名住院医师之间的329对教员和自我评估的委托分数。以教员分数作为金标准，我们将教员分数与住院医师分数（x-x）进行比较，并确定住院医师的准确性，包括过度自信和信心不足。
结果表明，住院医师在10.9%和54.4%的评估中分别存在过度自信和信心不足的情况，但他们个人的自我评估比教员评估者更一致（=0.70）。教员分数显著高于住院医师的自我评估分数（=0.396；z=4.39；<0.001）。作为宽松/温和的（=0.121，z=3.16，<0.01）和中立的（=0.137，z=3.57，<0.001）教员评估者，预测住院医师信心不足的可能性更高。与初级住院医师相比，高级住院医师信心不足的可能性显著更低（=-0.182，z=-2.45，<0.05）。在研究期间的两年内，自我评估的准确性没有显著变化。
大多数住院医师的自我评估不准确。我们的研究结果可能有助于确定这种不准确的来源。
