Department of Prosthodontics, Hadassah Medical Center, Faculty of Dental Medicine, Hebrew University of Jerusalem, Jerusalem, Israel.
Department of Otolaryngology/HNS, Hadassah Medical Center, Faculty of Medicine, Hebrew University of Jerusalem, Jerusalem, Israel.
BMC Med Educ. 2024 Mar 14;24(1):286. doi: 10.1186/s12909-024-05228-1.
Good technical skills are crucial for surgeons. Yet although surgical training programs strive to assess technical aptitude when selecting surgical residents, valid assessments of such aptitude are still lacking. Surgical simulators have been proposed as a potentially effective tool for this purpose. The current study aims to develop a technical aptitude test using a virtual reality surgical simulator, and to validate its use for the selection of surgical residents.
The study had three phases. In Phase 1, we developed an initial version of the technical aptitude test using the Lap-X-VR laparoscopic simulator. In Phases 2 and 3 we refined the test and collected empirical data to evaluate four main sources of validity evidence (content, response process, internal structure, and relationships with other variables), and to evaluate the feasibility and acceptability of the test. Specifically, Phase 2 comprised a review of the test by 30 senior surgeons, and in Phase 3 a revised version of the test was administered to 152 interns to determine its psychometric properties.
Both the surgeons and interns rated the test as highly relevant for selecting surgical residents. Analyses of the data obtained from the trial administration of the test supported the appropriateness of the score calculation process and showed good psychometric properties, including reliability (α = 0.83) and task discrimination (mean discrimination = 0.5, SD = 0.1). The correlations between test scores and background variables revealed significant correlations with gender, surgical simulator experience, and video game experience (ps < 0.001). These variables, however, explained together only 10% of the variance in test scores.
We describe the systematic development of an innovative virtual reality test for assessing technical aptitude in candidates for surgical training, and present evidence for its validity, feasibility and acceptability. Further validation is required to support the application of the test for selection, as well as to discern the impact of gender, surgical simulator experience, and video game experience on the fairness of test results. However, the test appears to be a promising tool that may help training programs assess the suitability of candidates for surgical training.
良好的技术技能对于外科医生至关重要。然而，尽管外科培训计划在选择外科住院医师时努力评估技术能力，但仍然缺乏这种能力的有效评估。外科模拟器已被提议作为一种潜在有效的工具。本研究旨在使用虚拟现实外科模拟器开发一种技术能力测试，并验证其用于选择外科住院医师的用途。
该研究分为三个阶段。在第 1 阶段，我们使用 Lap-X-VR 腹腔镜模拟器开发了技术能力测试的初始版本。在第 2 阶段和第 3 阶段，我们对测试进行了改进，并收集了经验数据，以评估四个主要的有效性证据来源（内容、反应过程、内部结构和与其他变量的关系），并评估测试的可行性和可接受性。具体来说，第 2 阶段包括 30 名资深外科医生对测试的审查，第 3 阶段则对 152 名实习生进行了修订后的测试，以确定其心理测量特性。
外科医生和实习生都认为该测试对于选择外科住院医师非常重要。对从测试实施中获得的数据进行分析支持得分计算过程的适当性，并显示出良好的心理测量特性，包括可靠性（α=0.83）和任务区分度（平均区分度=0.5，标准差=0.1）。测试分数与背景变量之间的相关性显示与性别、外科模拟器经验和视频游戏经验显著相关（p<0.001）。然而，这些变量共同仅解释了测试分数方差的 10%。
我们描述了一种评估外科培训候选人技术能力的创新虚拟现实测试的系统开发，并提供了其有效性、可行性和可接受性的证据。需要进一步验证来支持该测试在选择中的应用，以及辨别性别、外科模拟器经验和视频游戏经验对测试结果公平性的影响。然而，该测试似乎是一种有前途的工具，可以帮助培训计划评估候选人是否适合进行外科培训。
