Colorectal Surgery Unit, General Surgery Department, Marqués de Valdecilla University Hospital, Santander, Spain.
Valdecilla virtual Hospital, Valdecilla Biomedical Research Institute (IDIVAL), Santander, Spain.
BJS Open. 2022 May 2;6(3). doi: 10.1093/bjsopen/zrac041.
This study aimed to evaluate the use of binary metric-based (proficiency-based progression; PBP) performance assessments and global evaluative assessment of robotic skills (GEARS) of a robotic-assisted low anterior rectal resection (RA-LAR) procedure.
A prospective study of video analysis of RA-LAR procedures was carried out using the PBP metrics with binary parameters previously developed, and GEARS. Recordings were collected from five novice surgeons (≤30 RA-LAR previously performed) and seven experienced surgeons (>30 RA-LAR previously performed). Two consultant colorectal surgeons were trained to be assessors in the use of PBP binary parameters to evaluate the procedure phases, surgical steps, errors, and critical errors in male and female patients and GEARS scores. Novice and experienced surgeons were categorized and assessed using PBP metrics and GEARS; mean scores obtained were compared for statistical purpose. Also, the inter-rater reliability (IRR) of these assessment tools was evaluated.
Twenty unedited recordings of RA-LAR procedures were blindly assessed. Overall, using PBP metric-based assessment, a subgroup of experienced surgeons made more errors (20 versus 16, P = 0.158) and critical errors (9.2 versus 7.8, P = 0.417) than the novice group, although not significantly. However, during the critical phase of RA-LAR, experienced surgeons made significantly fewer errors than the novice group (95% CI of the difference, Lower = 0.104 - Upper = 5.155, df = 11.9, t = 2.23, p = 0.042), and a similar pattern was observed for critical errors. The PBP metric and GEARS assessment tools distinguished between the objectively assessed performance of experienced and novice colorectal surgeons performing RA-LAR (total error scores with PBP metrics, P = 0.019-0.008; GEARS scores, P = 0.029-0.025). GEARS demonstrated poor IRR (mean IRR 0.49) and weaker discrimination between groups (15-41 per cent difference). PBP binary metrics demonstrated good IRR (mean 0.94) and robust discrimination particularly for total error scores (58-64 per cent).
PBP binary metrics seem to be useful for metric-based training for surgeons learning RA-LAR procedures.
本研究旨在评估基于二进制指标（基于熟练程度的进展；PBP）的性能评估和机器人技能的整体评估（GEARS）在机器人辅助低位前直肠切除术（RA-LAR）中的应用。
对 RA-LAR 手术的视频分析进行前瞻性研究，使用先前开发的 PBP 指标和 GEARS 进行二进制参数。记录来自五名新手外科医生（≤30 例先前进行的 RA-LAR）和七名经验丰富的外科医生（>30 例先前进行的 RA-LAR）。两名顾问结直肠外科医生接受培训，以使用 PBP 二进制参数评估男性和女性患者以及 GEARS 评分的手术阶段、手术步骤、错误和关键错误。对新手和经验丰富的外科医生进行分类和评估，并使用 PBP 指标和 GEARS 进行评估；为了进行统计目的，比较了获得的平均分数。还评估了这些评估工具的内部评估者可靠性（IRR）。
对 20 次未经编辑的 RA-LAR 手术记录进行了盲法评估。总体而言，使用基于 PBP 度量的评估，经验丰富的外科医生组比新手组犯了更多的错误（20 比 16，P=0.158）和关键错误（9.2 比 7.8，P=0.417），尽管差异无统计学意义。然而，在 RA-LAR 的关键阶段，经验丰富的外科医生犯的错误明显少于新手组（差异的 95%置信区间下限=0.104-上限=5.155，df=11.9，t=2.23，p=0.042），并且对关键错误也观察到类似的模式。PBP 度量和 GEARS 评估工具在客观评估执行 RA-LAR 的经验丰富和新手结直肠外科医生的表现方面有所区别（PBP 度量的总错误评分，P=0.019-0.008；GEARS 评分，P=0.029-0.025）。GEARS 显示出较差的 IRR（平均 IRR 0.49）和对组间差异的较弱区分能力（15-41%差异）。PBP 二进制度量显示出良好的 IRR（平均 0.94）和强大的区分能力，特别是在总错误评分方面（58-64%差异）。
PBP 二进制指标似乎可用于外科医生学习 RA-LAR 手术的基于度量的培训。
