School of Medicine, Keele University, Keele, Staffordshire, UK
School of Medicine, Keele University, Keele, Staffordshire, UK.
BMJ Open. 2022 Dec 7;12(12):e064387. doi: 10.1136/bmjopen-2022-064387.
Objective structured clinical exams (OSCEs) are a cornerstone of assessing the competence of trainee healthcare professionals, but have been criticised for (1) lacking authenticity, (2) variability in examiners' judgements which can challenge assessment equivalence and (3) for limited diagnosticity of trainees' focal strengths and weaknesses. In response, this study aims to investigate whether (1) sharing integrated-task OSCE stations across institutions can increase perceived authenticity, while (2) enhancing assessment equivalence by enabling comparison of the standard of examiners' judgements between institutions using a novel methodology (video-based score comparison and adjustment (VESCA)) and (3) exploring the potential to develop more diagnostic signals from data on students' performances.
The study will use a complex intervention design, developing, implementing and sharing an integrated-task (research) OSCE across four UK medical schools. It will use VESCA to compare examiner scoring differences between groups of examiners and different sites, while studying how, why and for whom the shared OSCE and VESCA operate across participating schools. Quantitative analysis will use Many Facet Rasch Modelling to compare the influence of different examiners groups and sites on students' scores, while the operation of the two interventions (shared integrated task OSCEs; VESCA) will be studied through the theory-driven method of Realist evaluation. Further exploratory analyses will examine diagnostic performance signals within data.
The study will be extra to usual course requirements and all participation will be voluntary. We will uphold principles of informed consent, the right to withdraw, confidentiality with pseudonymity and strict data security. The study has received ethical approval from Keele University Research Ethics Committee. Findings will be academically published and will contribute to good practice guidance on (1) the use of VESCA and (2) sharing and use of integrated-task OSCE stations.
客观结构化临床考试（OSCE）是评估医学生培训师能力的基石，但一直受到以下批评：（1）缺乏真实性；（2）考官判断的可变性，这可能会挑战评估等效性；（3）对学员重点强项和弱点的诊断能力有限。针对这些问题，本研究旨在探讨：（1）在机构之间共享综合任务 OSCE 站是否可以提高感知真实性，同时（2）通过使用新的方法学（基于视频的评分比较和调整（VESCA））来比较机构之间考官判断的标准，从而增强评估等效性；（3）探索从学生表现数据中开发更多诊断信号的潜力。
该研究将采用复杂干预设计，在四所英国医学院开发、实施和共享一项综合任务（研究）OSCE。它将使用 VESCA 比较不同考官组和不同地点的考官评分差异，同时研究共享 OSCE 和 VESCA 在参与学校中的运作方式、原因和对象。定量分析将使用多方面 Rasch 建模来比较不同考官组和地点对学生成绩的影响，同时通过理论驱动的真实评估方法研究这两个干预措施（共享的综合任务 OSCE；VESCA）的运作。进一步的探索性分析将在数据中检查诊断性能信号。
该研究将作为额外的课程要求，所有参与都是自愿的。我们将坚持知情同意、退出权、保密与匿名、严格的数据安全等原则。该研究已获得基尔大学伦理委员会的批准。研究结果将在学术上发表，并将有助于关于（1）VESCA 的使用和（2）综合任务 OSCE 站的共享和使用的良好实践指南。
