Fried Marvin P, Sadoughi Babak, Weghorst Suzanne J, Zeltsan Michael, Cuellar Hernando, Uribe José I, Sasaki Clarence T, Ross Douglas A, Jacobs Joseph B, Lebowitz Richard A, Satava Richard M
Department of Otorhinolaryngology-Head and Neck Surgery, Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY 10467, USA.
Arch Otolaryngol Head Neck Surg. 2007 Apr;133(4):350-7. doi: 10.1001/archotol.133.4.350.
To establish discriminant validity of the endoscopic sinus surgery simulator (ES3) (Lockheed Martin, Akron, Ohio) between various health care provider experience levels and to define benchmarking criteria for skills assessment.
Prospective multi-institutional comparison study.
University-based tertiary care institution.
Ten expert otolaryngologists, 14 otolaryngology residents, and 10 medical students.
Subjects completed the ES3's virtual reality curriculum (10 novice mode, 10 intermediate mode, and 3 advanced mode trials). Performance scores were recorded on each trial. Performance differences were analyzed using analysis of variance for repeated measures (experience level as between-subjects factor).
Simulator performance scores, accuracy, time to completion, and hazard disruption.
The novice mode accurately distinguished the 3 groups, particularly at the onset of training (mean scores: senior otolaryngologists, 66.0; residents, 42.7; students, 18.3; for the paired comparisons between groups 1 and 2 and groups 1 and 3, P = .04 and .03, respectively). Subjects were not distinguished beyond trial 5. The intermediate mode only discriminated students from other subjects (P = .008). The advanced mode did not show performance differences between groups. Scores on the novice mode predicted those on the intermediate mode, which predicted advanced mode scores (r = 0.687), but no relationship was found between novice and advanced scores. All groups performed equally well and with comparable consistency at the outset of training. Expert scores were used to define benchmark criteria of optimal performance.
This study completes the construct validity assessment of the ES3 by demonstrating its discriminant capabilities. It establishes expert surgeon benchmark performance criteria and shows that the ES3 can train novice subjects to attain those. The refined analysis of trial performance scores could serve educational and skills assessment purposes. Current studies are evaluating the transfer of surgical skills acquired on the ES3 to the operating room (predictive validity).
确定内镜鼻窦手术模拟器（ES3，洛克希德·马丁公司，俄亥俄州阿克伦）在不同医疗服务提供者经验水平之间的判别效度，并定义技能评估的基准标准。
前瞻性多机构比较研究。
大学附属三级医疗机构。
10名耳鼻喉科专家、14名耳鼻喉科住院医师和10名医学生。
受试者完成ES3的虚拟现实课程（10次新手模式、10次中级模式和3次高级模式试验）。记录每次试验的表现分数。使用重复测量方差分析（经验水平作为组间因素）分析表现差异。
模拟器表现分数、准确性、完成时间和危险中断情况。
新手模式准确区分了3组，尤其是在训练开始时（平均分数：资深耳鼻喉科医生66.0；住院医师42.7；学生18.3；第1组和第2组以及第1组和第3组之间的配对比较，P分别为0.04和0.03）。第5次试验后各受试者之间没有差异。中级模式仅区分了学生与其他受试者（P = 0.008）。高级模式未显示组间表现差异。新手模式的分数可预测中级模式的分数，而中级模式的分数可预测高级模式的分数（r = 0.687），但新手和高级分数之间未发现相关性。在训练开始时，所有组的表现同样出色且一致性相当。专家分数用于定义最佳表现的基准标准。
本研究通过展示其判别能力完成了ES3的结构效度评估。它建立了专家外科医生的基准表现标准，并表明ES3可以训练新手受试者达到这些标准。对试验表现分数的精细分析可用于教育和技能评估目的。目前的研究正在评估在ES3上获得的外科技能向手术室的转化（预测效度）。
