Eva Kevin W, Regehr Glenn
Department of Clinical Epidemiology and Biostatistics, Program for Educational Research and Development, MDCL 3522, McMaster University, Hamilton, ON, L8N 3Z5, Canada.
Acad Med. 2007 Oct;82(10 Suppl):S81-4. doi: 10.1097/ACM.0b013e31813e6755.
Although self-assessment is widely acknowledged as a vital skill for members of self-regulating professions, a ubiquitous finding in the research literature is that self-ratings are quite poor when compared with externally generated measures of ability. Many researchers have identified this as a serious problem for the concept of self-regulation in the professions. However, we question the sufficiency of the operational definitions of self-assessment on which the previous research is based. This study examines the validity of a new conceptualization of self-assessment in practice and evaluates a series of measures for capturing self-assessment ability as defined by this new conceptualization.
Using a computer-delivered free-response test, the authors generated three measures intended to capture situational awareness: (1) response times to questions, (2) the ability to avoid responding to questions for which the respondent is less likely to be correct, and (3) the ability to select questions from content areas in which respondents have greater ability. In addition, the traditional measures of self-assessment (e.g., predictions of how many questions one would answer correctly) were administered.
Participants showed behavioral indications of being aware of the limits of their ability. They took longer to respond when their eventual answer was incorrect relative to when it was correct, they were able to avoid answering questions on which they were likely to be incorrect, and they selected content-based domains in an appropriate order given their accuracy.
These results provide evidence in favor of this new framework that should reorient the way in which self-assessment "skills" are conceptualized, taught, and evaluated in medical school and beyond.
尽管自我评估被广泛认为是自我监管职业成员的一项重要技能，但研究文献中一个普遍的发现是，与外部生成的能力衡量标准相比，自我评分相当差。许多研究人员将此视为职业自我监管概念的一个严重问题。然而，我们质疑先前研究所基于的自我评估操作定义是否充分。本研究检验了实践中自我评估新概念化的有效性，并评估了一系列用于捕捉由这一新概念化所定义的自我评估能力的测量方法。
作者使用计算机提供的自由回答测试，生成了三项旨在捕捉情境意识的测量方法：（1）对问题的回答时间，（2）避免回答受访者不太可能正确回答的问题的能力，以及（3）从受访者能力较强的内容领域中选择问题的能力。此外，还采用了传统的自我评估测量方法（例如，预测自己能正确回答多少问题）。
参与者表现出意识到自身能力局限的行为迹象。相对于回答正确时，当他们最终的答案错误时，回答时间更长，他们能够避免回答可能错误的问题，并且根据自身的准确性以适当的顺序选择基于内容的领域。
这些结果为这一新框架提供了支持证据，该框架应重新定位医学院及其他机构中自我评估“技能”的概念化、教学和评估方式。
