Harasym Peter H, Woloschuk Wayne, Cunning Leslie
Department of Community Health Sciences, University of Calgary, Calgary, AB, Canada.
Adv Health Sci Educ Theory Pract. 2008 Dec;13(5):617-32. doi: 10.1007/s10459-007-9068-0. Epub 2007 Jul 3.
Physician-patient communication is a clinical skill that can be learned and has a positive impact on patient satisfaction and health outcomes. A concerted effort at all medical schools is now directed at teaching and evaluating this core skill. Student communication skills are often assessed by an Objective Structure Clinical Examination (OSCE). However, it is unknown what sources of error variance are introduced into examinee communication scores by various OSCE components. This study primarily examined the effect different examiners had on the evaluation of students' communication skills assessed at the end of a family medicine clerkship rotation. The communication performance of clinical clerks from Classes 2005 and 2006 were assessed using six OSCE stations. Performance was rated at each station using the 28-item Calgary-Cambridge guide. Item Response Theory analysis using a Multifaceted Rasch model was used to partition the various sources of error variance and generate a "true" communication score where the effects of examiner, case, and items are removed. Variance and reliability of scores were as follows: communication scores (.20 and .87), examiner stringency/leniency (.86 and .91), case (.03 and .96), and item (.86 and .99), respectively. All facet scores were reliable (.87-.99). Examiner variance (.86) was more than four times the examinee variance (.20). About 11% of the clerks' outcome status shifted using "true" rather than observed/raw scores. There was large variability in examinee scores due to variation in examiner stringency/leniency behaviors that may impact pass-fail decisions. Exploring the benefits of examiner training and employing "true" scores generated using Item Response Theory analyses prior to making pass/fail decisions are recommended.
医患沟通是一项可以习得的临床技能，对患者满意度和健康结果有积极影响。现在所有医学院校都在齐心协力致力于教授和评估这项核心技能。学生的沟通技能通常通过客观结构化临床考试（OSCE）进行评估。然而，尚不清楚OSCE的各个组成部分会给考生的沟通分数引入哪些误差方差来源。本研究主要考察了不同考官对家庭医学实习轮转结束时评估的学生沟通技能的影响。使用六个OSCE考站对2005级和2006级临床实习医生的沟通表现进行评估。每个考站的表现根据28项卡尔加里-剑桥指南进行评分。使用多面Rasch模型进行项目反应理论分析，以划分各种误差方差来源，并生成一个去除了考官、病例和项目影响的“真实”沟通分数。分数的方差和信度分别如下：沟通分数（.20和.87）、考官的严格/宽松程度（.86和.91）、病例（.03和.96）以及项目（.86和.99）。所有方面的分数都是可靠的（.87 -.99）。考官方差（.86）是考生方差（.20）的四倍多。使用“真实”分数而非观察到的/原始分数时，约11%的实习医生的结果状态发生了变化。由于考官严格/宽松行为的差异，考生分数存在很大差异，这可能会影响及格/不及格的判定。建议探索考官培训的益处，并在做出及格/不及格判定之前采用通过项目反应理论分析生成的“真实”分数。
