Vanstrum Erik B, Ma Runzhuo, Maya-Silva Jacqueline, Sanford Daniel, Nguyen Jessica H, Lei Xiaomeng, Chevinksy Michael, Ghoreifi Alireza, Han Jullet, Polotti Charles F, Powers Ryan, Yip Wesley, Zhang Michael, Aron Monish, Collins Justin, Daneshmand Siamak, Davis John W, Desai Mihir M, Gerjy Roger, Goh Alvin C, Kimmig Rainer, Lendvay Thomas S, Porter James, Sotelo Rene, Sundaram Chandru P, Cen Steven, Gill Inderbir S, Hung Andrew J
Catherine & Joseph Aresty Department of Urology, Center for Robotic Simulation & Education, University of Southern California Institute of Urology, Los Angeles, CA, USA.
Department of Radiology, University of Southern California, Los Angeles, CA, USA.
Urol Pract. 2021 Sep;8(5):596-604. doi: 10.1097/upj.0000000000000246.
Evaluation of surgical competency has important implications for training new surgeons, accreditation, and improving patient outcomes. A method to specifically evaluate dissection performance does not yet exist. This project aimed to design a tool to assess surgical dissection quality.
Delphi method was used to validate structure and content of the dissection evaluation. A multi-institutional and multi-disciplinary panel of 14 expert surgeons systematically evaluated each element of the dissection tool. Ten blinded reviewers evaluated 46 de-identified videos of pelvic lymph node and seminal vesicle dissections during the robot-assisted radical prostatectomy. Inter-rater variability was calculated using prevalence-adjusted and bias-adjusted kappa. The area under the curve from receiver operating characteristic curve was used to assess discrimination power for overall DART scores as well as domains in discriminating trainees (≤100 robotic cases) from experts (>100).
Four rounds of Delphi method achieved language and content validity in 27/28 elements. Use of 3- or 5-point scale remained contested; thus, both scales were evaluated during validation. The 3-point scale showed improved kappa for each domain. Experts demonstrated significantly greater total scores on both scales (3-point, < 0.001; 5-point, < 0.001). The ability to distinguish experience was equivalent for total score on both scales (3-point AUC= 0.92, CI 0.82-1.00, 5-point AUC= 0.92, CI 0.83-1.00).
We present the development and validation of Dissection Assessment for Robotic Technique (DART), an objective and reproducible 3-point surgical assessment to evaluate tissue dissection. DART can effectively differentiate levels of surgeon experience and can be used in multiple surgical steps.
评估手术能力对培训新外科医生、认证及改善患者预后具有重要意义。目前尚不存在专门评估解剖操作表现的方法。本项目旨在设计一种评估手术解剖质量的工具。
采用德尔菲法验证解剖评估的结构和内容。由14名专家外科医生组成的多机构、多学科小组系统地评估了解剖工具的每个要素。10名盲法评审员对机器人辅助根治性前列腺切除术中46段去识别化的盆腔淋巴结和精囊解剖视频进行了评估。使用患病率调整和偏差调整的kappa计算评分者间的变异性。采用受试者操作特征曲线下面积评估整体DART评分以及区分低年资学员（机器人手术病例≤100例）和专家（机器人手术病例>100例）的各领域的鉴别能力。
经过四轮德尔菲法，27/28个要素实现了语言和内容效度。对于采用3分制还是5分制仍存在争议；因此，在验证过程中对两种评分制都进行了评估。3分制在每个领域的kappa值均有所提高。专家在两种评分制下的总分均显著更高（3分制，P<0.001；5分制，P<0.001）。两种评分制在区分经验方面的能力相当（3分制AUC=0.92，CI 0.82 - 1.00；5分制AUC=0.92，CI 0.83 - 1.00）。
我们展示了机器人技术解剖评估（DART）的开发与验证，这是一种用于评估组织解剖的客观且可重复的3分制手术评估方法。DART能够有效区分外科医生的经验水平，可用于多个手术步骤。
