University of Bern, Institute for Medical Education, Department for Assessment and Evaluation, Bern, Switzerland.
University of Bern, Institute for Medical Education, Department for Software Development, Usability Consulting and IT Infrastructure, Bern, Switzerland.
GMS J Med Educ. 2022 Apr 14;39(2):Doc24. doi: 10.3205/zma001545. eCollection 2022.
Only a few studies with small sample sizes have compared electronic Objective Structured Clinical Examination (OSCE) rating checklists with traditional paper-based OSCE rating checklists. In this study, the examiner-perceived usability and preference for type of OSCE checklist (electronic vs. paper based) were compared, and the influence of OSCE checklist type on missed ratings was determined, for the Swiss Federal Licensing Examination in clinical skills for human medicine.
All examiners in the Swiss Federal Licensing Examination in clinical skills for human medicine were invited over two subsequent years to evaluate the OSCE checklist type they had worked with during the examination. This was based on a questionnaire with 14 closed questions (i.e., demographic, checklist-type experience, perceived usability, checklist type preference). Furthermore, the numbers of missed ratings for the paper-based checklist were recorded.
The data from these examiners (=377) with experience of both OSCE checklist types were analyzed. The electronic OSCE checklist was rated significantly higher on all usability aspects (i.e., ease of use, candidate rating and error correction, clarity, distraction using the checklist, overall satisfaction), except for the speed of registering comments (no significant difference). The majority of the examiners in both years (2014: 54.5%, =60, 2015: 89.8%, =230) reported preference for working with the electronic OSCE checklist in the future. Missed ratings were seen for 14.2% of the paper-based OSCE checklists, which were prevented with the electronic OSCE checklists.
Electronic OSCE checklists were rated significantly more user-friendly and were preferred over paper-based OSCE checklists by a broad national sample of examiners, supporting previous results from faculty-level examinations. Furthermore, missed ratings were prevented with the electronic OSCE checklists. Overall, the use of electronic OSCE checklists is therefore advisable.
仅有少数小样本量的研究比较了电子客观结构化临床考试（OSCE）评分检查表与传统纸质 OSCE 评分检查表。在这项研究中，我们比较了瑞士联邦医师执照临床技能考试中检查者对 OSCE 检查表类型（电子与纸质）的感知可用性和偏好，并确定了 OSCE 检查表类型对漏评的影响。
在接下来的两年中，瑞士联邦医师执照临床技能考试中的所有考官都被邀请评估他们在考试中使用的 OSCE 检查表类型。这是基于一份包含 14 个封闭问题的问卷（即人口统计学、检查表类型经验、感知可用性、检查表类型偏好）。此外，还记录了纸质检查表的漏评次数。
对具有两种 OSCE 检查表类型经验的这些考官（=377）的数据进行了分析。在所有可用性方面（即使用方便性、考生评分和错误更正、清晰度、使用检查表时的分心、总体满意度），电子 OSCE 检查表的评分均显著更高，除了记录评论的速度（无显著差异）。在这两年中（2014 年：54.5%，=60；2015 年：89.8%，=230），大多数考官表示未来更喜欢使用电子 OSCE 检查表。纸质 OSCE 检查表有 14.2%的漏评，而电子 OSCE 检查表则可以避免这种情况。
电子 OSCE 检查表的用户友好性得到了更广泛的全国性考官样本的高度评价，并且比纸质 OSCE 检查表更受欢迎，这支持了之前在教师层面考试中的结果。此外，电子 OSCE 检查表还可以防止漏评。总体而言，使用电子 OSCE 检查表是可取的。
