Read Emma K, Bell Catriona, Rhind Susan, Hecker Kent G
Department of Veterinary Clinical and Diagnostic Sciences, University of Calgary Faculty of Veterinary Medicine, Calgary, Alberta, Canada.
Royal (Dick) School of Veterinary Studies, University of Edinburgh, Roslin, Midlothian, Scotland.
PLoS One. 2015 Mar 30;10(3):e0121000. doi: 10.1371/journal.pone.0121000. eCollection 2015.
OSCEs (Objective Structured Clinical Examinations) are widely used in health professions to assess clinical skills competence. Raters use standardized binary checklists (CL) or multi-dimensional global rating scales (GRS) to score candidates performing specific tasks. This study assessed the reliability of CL and GRS scores in the assessment of veterinary students, and is the first study to demonstrate the reliability of GRS within veterinary medical education. Twelve raters from two different schools (6 from University of Calgary [UCVM] and 6 from Royal (Dick) School of Veterinary Studies [R(D)SVS] were asked to score 12 students (6 from each school). All raters assessed all students (video recordings) during 4 OSCE stations (bovine haltering, gowning and gloving, equine bandaging and skin suturing). Raters scored students using a CL, followed by the GRS. Novice raters (6 R(D)SVS) were assessed independently of expert raters (6 UCVM). Generalizability theory (G theory), analysis of variance (ANOVA) and t-tests were used to determine the reliability of rater scores, assess any between school differences (by student, by rater), and determine if there were differences between CL and GRS scores. There was no significant difference in rater performance with use of the CL or the GRS. Scores from the CL were significantly higher than scores from the GRS. The reliability of checklist scores were .42 and .76 for novice and expert raters respectively. The reliability of the global rating scale scores were .7 and .86 for novice and expert raters respectively. A decision study (D-study) showed that once trained using CL, GRS could be utilized to reliably score clinical skills in veterinary medicine with both novice and experienced raters.
客观结构化临床考试（OSCEs）在卫生专业领域被广泛用于评估临床技能能力。评分者使用标准化的二元检查表（CL）或多维度整体评分量表（GRS）对执行特定任务的考生进行评分。本研究评估了CL和GRS评分在评估兽医专业学生中的可靠性，并且是第一项在兽医医学教育中证明GRS可靠性的研究。来自两所不同学校的12名评分者（卡尔加里大学[UCVM]的6名和皇家（迪克）兽医学院[R(D)SVS]的6名）被要求对12名学生（每所学校6名）进行评分。所有评分者在4个OSCE站点（牛的套索、穿手术衣和戴手套、马的包扎和皮肤缝合）期间对所有学生（录像）进行评估。评分者先用CL对学生进行评分，然后再用GRS评分。新手评分者（6名R(D)SVS）与专家评分者（6名UCVM）独立进行评估。使用概化理论（G理论）、方差分析（ANOVA）和t检验来确定评分者分数的可靠性，评估学校之间的任何差异（按学生、按评分者），并确定CL和GRS分数之间是否存在差异。使用CL或GRS时，评分者表现没有显著差异。CL的分数显著高于GRS的分数。新手评分者和专家评分者的检查表分数可靠性分别为0.42和0.76。新手评分者和专家评分者的整体评分量表分数可靠性分别为0.7和0.86。一项决策研究（D研究）表明，一旦使用CL进行培训，GRS就可以用于新手和经验丰富的评分者对兽医临床技能进行可靠评分。
