USC Institute of Urology, Hillard and Roclyn Herzog Center for Robotic Surgery, Keck School of Medicine, University of Southern California, Los Angeles, CA.
BJU Int. 2013 Oct;112(6):864-71. doi: 10.1111/bju.12045. Epub 2013 Mar 7.
To evaluate three standardized robotic surgery training methods, inanimate, virtual reality and in vivo, for their construct validity. To explore the concept of cross-method validity, where the relative performance of each method is compared.
Robotic surgical skills were prospectively assessed in 49 participating surgeons who were classified as follows: 'novice/trainee': urology residents, previous experience <30 cases (n = 38) and 'experts': faculty surgeons, previous experience ≥30 cases (n = 11). Three standardized, validated training methods were used: (i) structured inanimate tasks; (ii) virtual reality exercises on the da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA, USA); and (iii) a standardized robotic surgical task in a live porcine model with performance graded by the Global Evaluative Assessment of Robotic Skills (GEARS) tool. A Kruskal-Wallis test was used to evaluate performance differences between novices and experts (construct validity). Spearman's correlation coefficient (ρ) was used to measure the association of performance across inanimate, simulation and in vivo methods (cross-method validity).
Novice and expert surgeons had previously performed a median (range) of 0 (0-20) and 300 (30-2000) robotic cases, respectively (P < 0.001). Construct validity: experts consistently outperformed residents with all three methods (P < 0.001). Cross-method validity: overall performance of inanimate tasks significantly correlated with virtual reality robotic performance (ρ = -0.7, P < 0.001) and in vivo robotic performance based on GEARS (ρ = -0.8, P < 0.0001). Virtual reality performance and in vivo tissue performance were also found to be strongly correlated (ρ = 0.6, P < 0.001).
We propose the novel concept of cross-method validity, which may provide a method of evaluating the relative value of various forms of skills education and assessment. We externally confirmed the construct validity of each featured training tool.
评估三种标准化机器人手术培训方法（非生物、虚拟现实和活体）的结构有效性。探索交叉方法有效性的概念，即比较每种方法的相对性能。
前瞻性评估 49 名参与研究的外科医生的机器人手术技能，将他们分为以下两类：“新手/学员”：泌尿科住院医师，经验<30 例（n=38）和“专家”：教员外科医生，经验≥30 例（n=11）。使用三种标准化、经过验证的培训方法：（i）结构化非生物任务；（ii）在达芬奇技能模拟器（直觉外科公司，加利福尼亚州森尼韦尔）上进行虚拟现实练习；以及（iii）在活体猪模型中进行标准化机器人手术任务，使用全球机器人技能评估工具（GEARS）进行性能分级。使用 Kruskal-Wallis 检验评估新手和专家之间的表现差异（结构有效性）。使用 Spearman 相关系数（ρ）衡量非生物、模拟和活体方法之间的性能相关性（交叉方法有效性）。
新手和专家外科医生分别在机器人手术方面的经验中位数（范围）为 0（0-20）和 300（30-2000）例（P<0.001）。结构有效性：专家使用所有三种方法均始终优于住院医师（P<0.001）。交叉方法有效性：非生物任务的整体表现与虚拟现实机器人表现（ρ=-0.7，P<0.001）和基于 GEARS 的活体机器人表现（ρ=-0.8，P<0.0001）显著相关。虚拟现实表现和活体组织表现之间也存在很强的相关性（ρ=0.6，P<0.001）。
我们提出了交叉方法有效性的新概念，这可能为评估各种形式的技能教育和评估的相对价值提供一种方法。我们外部证实了每种特色培训工具的结构有效性。
