Medical Education Centre, School of Medicine, University of Nottingham, Nottingham, UK.
School of Education, Virginia Commonwealth University, Richmond, VA, USA.
Med Teach. 2022 Jun;44(6):582-595. doi: 10.1080/0142159X.2021.1993164. Epub 2021 Nov 2.
The ratings that judges or examiners use for determining pass marks and students' performance on OSCEs serve a number of essential functions in medical education assessment, and their validity is a pivotal issue. However, some types of errors often occur in ratings that require special efforts to minimise. Rater characteristics (e.g. generosity error, severity error, central tendency error or halo error) may present a source of performance irrelevant variance. Prior literature shows the fundamental problems in student performance measurement attached to judges' or examiners' errors. It also indicates that the control of such errors supports a robust and credible pass mark and thus, accurate student marks. Therefore, for a standard-setter who identifies the pass mark and an examiner who rates student performance in OSCEs, proper, user-friendly feedback on their standard-setting and ratings is essential for reducing bias. This feedback provides useful avenues for understanding why performance ratings may be irregular and how to improve the quality of ratings. This AMEE Guide discusses various methods of feedback to support examiners' understanding of the performance of students and the standard-setting process with an effort to make inferences from assessments fair, valid and reliable.
在医学教育评估中，评判者或考官用于确定及格分数和学生在客观结构化临床考试（OSCE）中表现的评分标准具有多项重要功能，其有效性是一个关键问题。然而，评分中经常会出现一些类型的错误，需要特别努力加以最小化。评分者的特征（例如宽厚误差、严厉误差、趋中误差或晕轮误差）可能是导致与绩效无关的变异的一个来源。先前的文献表明，与评判者或考官的错误相关联的学生表现衡量存在根本问题。它还表明，此类错误的控制有助于确定可靠且可信的及格分数，从而准确地给出学生的分数。因此，对于确定及格分数的标准制定者和在 OSCE 中对学生表现进行评分的考官来说，为他们提供关于标准制定和评分的适当、易于使用的反馈对于减少偏差至关重要。这种反馈为理解为什么绩效评分可能不规律以及如何提高评分质量提供了有用的途径。本 AMEE 指南讨论了各种反馈方法，以支持考官理解学生的表现和标准制定过程，努力使评估的推断公平、有效和可靠。
