Orsi Academy, Proefhoevestraat 12, Melle, 9090, Ghent, Belgium.
Department of Urology, Onze-Lieve-Vrouw Ziekenhuis, Aalst, Belgium.
J Robot Surg. 2023 Aug;17(4):1401-1409. doi: 10.1007/s11701-023-01521-1. Epub 2023 Jan 23.
RAPN training usually takes place in-vivo and methods vary across countries/institutions. No common system exists to objectively assess trainee capacity to perform RAPN at predetermined performance levels prior to in-vivo practice. The identification of objective performance metrics for RAPN training is a crucial starting point to improve training and surgical outcomes. The authors sought to examine the reliability, construct and discriminative validity of objective intraoperative performance metrics which best characterize the optimal and suboptimal performance of a reference approach for training novice RAPN surgeons. Seven Novice and 9 Experienced RAPN surgeons video recorded one or two independently performed RAPN procedures in the human. The videos were anonymized and two experienced urology surgeons were trained to reliably score RAPN performance, using previously developed metrics. The assessors were blinded to the performing surgeon, hospital and surgeon group. They independently scored surgeon RAPN performance. Novice and Experienced group performance scores were compared for procedure steps completed and errors made. Each group was divided at the median for Total Errors score, and subgroup scores (i.e., Novice HiErrs and LoErrs, Experienced HiErrs and LoErrs) were compared. The mean inter-rater reliability (IRR) for scoring was 0.95 (range 0.84-1). Compared with Novices, Experienced RAPN surgeons made 69% fewer procedural Total Errors. This difference was accentuated when the LoErr Expert RAPN surgeon's performance was compared with the HiErrs Novice RAPN surgeon's performance with an observed 170% fewer Total Errors. GEARS showed poor reliability (Mean IRR = 0.44; range 0.0-0.8), for scoring RAPN surgical performance. The RAPN procedure metrics reliably distinguish Novice and Experienced surgeon performances. They further differentiated performance levels within a group with similar experiences. Reliable and valid metrics will underpin quality-assured novice RAPN surgical training.
RAPN 培训通常在体内进行，方法因国家/机构而异。在体内实践之前，没有共同的系统来客观评估学员在预定绩效水平下进行 RAPN 的能力。确定 RAPN 培训的客观绩效指标对于提高培训和手术结果至关重要。作者旨在研究客观术中绩效指标的可靠性、构建和判别有效性，这些指标最好地描述了培训新手 RAPN 外科医生的参考方法的最佳和次优表现。7 名新手和 9 名经验丰富的 RAPN 外科医生在人体上独立进行了一次或两次 RAPN 手术，并对手术过程进行了录像。视频被匿名化，两名经验丰富的泌尿科外科医生接受了培训，以使用先前开发的指标可靠地对 RAPN 手术表现进行评分。评估员对手术医生、医院和外科医生小组一无所知。他们独立地对医生的 RAPN 手术表现进行评分。比较了新手和经验丰富组完成的手术步骤和错误。将每组的总错误分数中位数进行分组，比较亚组分数（即新手高错误分数和低错误分数、经验丰富的高错误分数和低错误分数）。评分的平均组内相关系数（IRR）为 0.95（范围 0.84-1）。与新手相比，经验丰富的 RAPN 外科医生的手术总错误减少了 69%。当比较经验丰富的 LoErr 专家 RAPN 外科医生的表现与新手的 HiErrs 外科医生的表现时，这一差异更加明显，总错误减少了 170%。GEARS 评分 RAPN 手术表现的可靠性较差（平均 IRR=0.44；范围 0.0-0.8）。RAPN 手术指标可靠地区分新手和经验丰富的外科医生的表现。它们进一步区分了具有相似经验的组内的绩效水平。可靠和有效的指标将为有保证的新手 RAPN 手术培训提供支持。
