Andreatta Pamela B, Patel Jigarkumar A, Buzzelli Mark D, Nelson Kenneth J, Graybill John Christopher, Jensen Shane D, Remick Kyle N, Bowyer Mark W, Gurney Jennifer M
From the Department of Surgery, Uniformed Services University of the Health Science and the Walter Reed National Military Medical Center, Bethesda, MD.
Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD.
Ann Surg Open. 2022 Jun 29;3(3):e180. doi: 10.1097/AS9.0000000000000180. eCollection 2022 Sep.
The research question asked to what extent do self-rated performance scores of individual surgeons correspond to assessed procedural performance abilities and to peer ratings of procedural performance during a mass casualty (MASCAL) event?
Self-assessment using performance rating scales is ubiquitous in surgical education as a proxy for direct measurement of competence. The validity and reliability of self-ratings as competency measures are susceptible to cognitive biases such as Dunning-Kruger effects, which describe how individuals over/underestimate their own performance compared to assessments from independent sources. The ability of surgeons to accurately self-assess their procedural performance remains undetermined.
A purposive sample of military surgeons (N = 13) who collectively cared for trauma patients during a MASCAL event participated in the study. Pre-event performance assessment scores for 32 trauma procedures were compared with post-event self and peer performance ratings using tests ( < 0.05) and effect sizes (Cohen's ).
There were no significant differences between peer ratings and performance assessment scores. There were significant differences between self-ratings and both peer ratings ( < 0.001) and performance assessment scores ( < 0.001). Effect sizes were very large for self to peer rating comparison (Cohen's = 2.34) and self to performance assessment comparison (Cohen's = 2.77).
The outcomes demonstrate that self-ratings were significantly lower than the independently determined assessment scores for each surgeon, revealing a Dunning-Kruger effect for highly skilled individuals underestimating their abilities. These outcomes underscore the limitations of self-assessment for measuring competence.
本研究问题是，在大规模伤亡事件（MASCAL）中，个体外科医生的自评表现得分与评估的手术操作能力以及手术操作表现的同行评分在多大程度上相符？
在外科教育中，使用表现评分量表进行自我评估作为直接衡量能力的替代方法很普遍。自我评分作为能力衡量指标的有效性和可靠性容易受到认知偏差的影响，如邓宁-克鲁格效应，该效应描述了个体与独立来源的评估相比，如何高估/低估自己的表现。外科医生准确自我评估其手术操作表现的能力仍未确定。
选取13名在大规模伤亡事件中共同护理创伤患者 的军队外科医生作为有目的的样本参与研究 。使用检验（<0.05）和效应量（科恩效应量），将32项创伤手术的术前表现评估得分与术后自我和同行表现评分进行比较 。
同行评分与表现评估得分之间无显著差异 自我评分与同行评分（<0.001）和表现评估得分（<0.001）之间均存在显著差异 自我与同行评分比较的效应量非常大（科恩效应量=2.34），自我与表现评估比较的效应量也非常大（科恩效应量=2.77）。
结果表明，每位外科医生的自我评分显著低于独立确定的评估得分 ，这表明高技能个体存在低估自身能力的邓宁-克鲁格效应。这些结果强调了自我评估在衡量能力方面的局限性。
