Yeates Peter, Sebok-Syer Stefanie S
a School of Medicine, Keele University , Stoke-on-Trent , UK.
b Faculty of Education, Queen's University, Kingston , Ontario , Canada.
Med Teach. 2017 Jan;39(1):92-99. doi: 10.1080/0142159X.2017.1248916. Epub 2016 Nov 29.
OSCEs are commonly conducted in multiple cycles (different circuits, times, and locations), yet the potential for students' allocation to different OSCE cycles is rarely considered as a source of variance-perhaps in part because conventional psychometrics provide limited insight.
We used Many Facet Rasch Modeling (MFRM) to estimate the influence of "examiner cohorts" (the combined influence of the examiners in the cycle to which each student was allocated) on students' scores within a fully nested multi-cycle OSCE.
Observed average scores for examiners cycles varied by 8.6%, but model-adjusted estimates showed a smaller range of 4.4%. Most students' scores were only slightly altered by the model; the greatest score increase was 5.3%, and greatest score decrease was -3.6%, with 2 students passing who would have failed.
Despite using 16 examiners per cycle, examiner variability did not completely counter-balance, resulting in an influence of OSCE cycles on students' scores. Assumptions were required for the MFRM analysis; innovative procedures to overcome these limitations and strengthen OSCEs are discussed.
OSCE cycle allocation has the potential to exert a small but unfair influence on students' OSCE scores; these little-considered influences should challenge our assumptions and design of OSCEs.
客观结构化临床考试（OSCE）通常以多个轮次进行（不同的考站、时间和地点），然而学生被分配到不同OSCE轮次的可能性很少被视为方差来源——部分原因可能是传统心理测量学提供的见解有限。
我们使用多面Rasch模型（MFRM）来估计“考官群组”（分配给每个学生的轮次中考官的综合影响）对完全嵌套的多轮OSCE中学生分数的影响。
考官轮次的观察平均分数相差8.6%，但模型调整后的估计显示范围较小，为4.4%。大多数学生的分数仅被模型轻微改变；分数最大增幅为5.3%，最大降幅为-3.6%，有2名原本会不及格的学生通过了考试。
尽管每个轮次使用16名考官，但考官的变异性并未完全抵消，导致OSCE轮次对学生分数产生影响。MFRM分析需要一些假设；讨论了克服这些限制并加强OSCE的创新程序。
OSCE轮次分配有可能对学生的OSCE分数产生微小但不公平的影响；这些很少被考虑的影响应该挑战我们对OSCE的假设和设计。
