Mühling Tobias, Schreiner Verena, Appel Marc, Leutritz Tobias, König Sarah
Institute of Medical Teaching and Medical Education Research, University Hospital Würzburg, Würzburg, Germany.
J Med Internet Res. 2025 Jan 10;27:e55066. doi: 10.2196/55066.
Objective structured clinical examinations (OSCEs) are a widely recognized and accepted method to assess clinical competencies but are often resource-intensive.
This study aimed to evaluate the feasibility and effectiveness of a virtual reality (VR)-based station (VRS) compared with a traditional physical station (PHS) in an already established curricular OSCE.
Fifth-year medical students participated in an OSCE consisting of 10 stations. One of the stations, emergency medicine, was offered in 2 modalities: VRS and PHS. Students were randomly assigned to 1 of the 2 modalities. We used 2 distinct scenarios to prevent content leakage among participants. Student performance and item characteristics were analyzed, comparing the VRS with PHS as well as with 5 other case-based stations. Student perceptions of the VRS were collected through a quantitative and qualitative postexamination online survey, which included a 5-point Likert scale ranging from 1 (minimum) to 5 (maximum), to evaluate the acceptance and usability of the VR system. Organizational and technical feasibility as well as cost-effectiveness were also evaluated.
Following randomization and exclusions of invalid data sets, 57 and 66 participants were assessed for the VRS and PHS, respectively. The feasibility evaluation demonstrated smooth implementation of both VR scenarios (septic and anaphylactic shock) with 93% (53/57) of students using the VR technology without issues. The difficulty levels of the VRS scenarios (septic shock: P=.67; anaphylactic shock: P=.58) were comparable to the average difficulty of all stations (P=.68) and fell within the reference range (0.4-0.8). In contrast, VRS demonstrated above-average values for item discrimination (septic shock: r'=0.40; anaphylactic shock: r'=0.33; overall r'=0.30; with values >0.3 considered good) and discrimination index (septic shock: D=0.25; anaphylactic shock: D=0.26; overall D=0.16, with 0.2-0.3 considered mediocre and <0.2 considered poor). Apart from some hesitancy toward its broader application in future practical assessments (mean 3.07, SD 1.37 for VRS vs mean 3.65, SD 1.18 for PHS; P=.03), there were no other differences in perceptions between VRS and PHS. Thematic analysis highlighted the realistic portrayal of medical emergencies and fair assessment conditions provided by the VRS. Regarding cost-effectiveness, initial development of the VRS can be offset by long-term savings in recurring expenses like standardized patients and consumables.
Integration of the VRS into the current OSCE framework proved feasible both technically and organizationally, even within the strict constraints of short examination phases and schedules. The VRS was accepted and positively received by students across various levels of technological proficiency, including those with no prior VR experience. Notably, the VRS demonstrated comparable or even superior item characteristics, particularly in terms of discrimination power. Although challenges remain, such as technical reliability and some acceptance concerns, VR remains promising in applications of clinical competence assessment.
客观结构化临床考试（OSCE）是一种广泛认可和接受的评估临床能力的方法，但通常资源消耗大。
本研究旨在评估在已确立的课程OSCE中，基于虚拟现实（VR）的考站（VRS）与传统实体考站（PHS）相比的可行性和有效性。
五年级医学生参加了一个由10个考站组成的OSCE。其中一个考站，即急诊医学，以两种形式提供：VRS和PHS。学生被随机分配到两种形式中的一种。我们使用了2种不同的场景以防止参与者之间的内容泄露。分析了学生的表现和题目特征，将VRS与PHS以及其他5个基于病例的考站进行比较。通过定量和定性的考后在线调查收集学生对VRS的看法，该调查包括一个从1（最低）到5（最高）的5点李克特量表，以评估VR系统的可接受性和可用性。还评估了组织和技术可行性以及成本效益。
在随机分组并排除无效数据集后，分别对57名和66名参与者进行了VRS和PHS评估。可行性评估表明，两种VR场景（脓毒性休克和过敏性休克）均顺利实施，93%（53/57）的学生使用VR技术没有问题。VRS场景的难度水平（脓毒性休克：P = 0.67；过敏性休克：P = 0.58）与所有考站的平均难度（P = 0.68）相当，且在参考范围内（0.4 - 0.8）。相比之下，VRS在题目区分度（脓毒性休克：r' = 0.40；过敏性休克：r' = 0.33；总体r' = 0.30；值>0.3被认为良好）和区分指数（脓毒性休克：D = 0.25；过敏性休克：D = 0.26；总体D = 0.16，0.2 - 0.3被认为中等，<0.2被认为较差）方面表现出高于平均水平的值。除了对其在未来实际评估中更广泛应用存在一些犹豫（VRS的平均分为3.07，标准差为1.37，PHS的平均分为3.65，标准差为1.18；P = 0.03）外，VRS和PHS在看法上没有其他差异。主题分析强调了VRS对医疗紧急情况的逼真描绘和公平的评估条件。在成本效益方面，VRS的初始开发成本可以通过长期节省标准化病人和消耗品等经常性费用来抵消。
将VRS整合到当前的OSCE框架中，在技术和组织上都被证明是可行的，即使在考试阶段和日程安排严格的限制下也是如此。VRS被不同技术水平的学生接受并给予积极评价，包括那些没有VR经验的学生。值得注意的是，VRS表现出相当甚至更优的题目特征，特别是在区分能力方面。尽管挑战仍然存在，如技术可靠性和一些接受度方面的问题，但VR在临床能力评估应用中仍具有前景。
