Department of Surgery, McGill University, Montreal, QC, Canada.
Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, Montreal, QC, Canada.
Surg Endosc. 2023 Mar;37(3):2281-2289. doi: 10.1007/s00464-022-09466-6. Epub 2022 Aug 3.
Self-review of recorded surgical procedures offers new opportunities for trainees to extend technical learning outside the operating-room. Valid tools for self-assessment are required prior to evaluating the effectiveness of video-review in enhancing technical learning. Therefore, we aimed to contribute evidence regarding the validity of intraoperative performance assessment tools for video-based self-assessment by general surgery trainees when performing laparoscopic cholecystectomies.
Using a web-based platform, general surgery trainees in a university-based residency program submitted recorded laparoscopic cholecystectomy procedures where they acted as the supervised primary surgeon. Attending surgeons measured operative performance at the time of surgery using general and procedure-specific assessment tools [Global Operative Assessment of Laparoscopic Skills (GOALS) and Operative Performance Rating System (OPRS), respectively] and entrustability level (O-SCORE). Trainees self-evaluated their performance from video-review using the same instruments. The validity of GOALS and OPRS for trainee self-assessment was investigated by testing the hypotheses that self-assessment scores correlate with (H1) expert assessment scores, (H2) O-SCORE, and (H3) procedure time and that (H4) self-assessment based on these instruments differentiates junior [postgraduate year (PGY) 1-3] and senior trainees (PGY 4-5), as well as (H5)simple [Visual Analogue Scale (VAS) ≤ 4] versus complex cases (VAS > 4). All hypotheses were based on previous literature, defined a priori, and were tested according to the COSMIN consensus on measurement properties.
A total of 35 videos were submitted (45% female and 45% senior trainees) and self-assessed. Our data supported 2 out of 5 hypotheses (H1 and H4) for GOALS and 3 out of 5 hypotheses (H1, H4 and H5) for OPRS, for trainee self-assessment.
OPRS, a procedure-specific assessment tool, was better able to differentiate between groups expected to have different levels of intraoperative performance, compared to GOALS, a general assessment tool. Given the interest in video-based learning, there is a need to further develop valid procedure-specific tools to support video-based self-assessment by trainees in a range of procedures.
对已记录手术过程的自我审查为学员提供了在手术室外扩展技术学习的新机会。在评估视频审查在增强技术学习方面的有效性之前，需要有用于自我评估的有效工具。因此，我们旨在提供有关一般外科住院医师在进行腹腔镜胆囊切除术时，使用基于视频的自我评估进行术中绩效评估工具的有效性的证据。
使用基于网络的平台，大学附属医院住院医师计划中的普通外科住院医师提交了记录腹腔镜胆囊切除术的手术过程，他们在这些手术中担任受监督的主刀医师。主治外科医生在手术时使用一般和特定于程序的评估工具（分别为全球腹腔镜技能操作评估量表（GOALS）和操作绩效评分系统（OPRS））以及可信赖度评分（O-SCORE）来测量手术操作的表现。住院医师从视频审查中自我评估自己的表现，使用相同的工具。通过检验以下假设来研究 GOALS 和 OPRS 用于住院医师自我评估的有效性：自我评估分数与（H1）专家评估分数，（H2）O-SCORE，和（H3）手术时间相关；以及（H4）基于这些工具的自我评估可以区分初级（研究生第 1-3 年）和高级住院医师（研究生第 4-5 年），以及（H5）简单（视觉模拟量表（VAS）≤4）与复杂病例（VAS>4）。所有假设均基于先前的文献，预先定义，并根据 COSMIN 测量特性共识进行检验。
共提交了 35 个视频（45％为女性，45％为高级住院医师）并进行了自我评估。我们的数据支持 GOALS 的 5 个假设中的 2 个（H1 和 H4）和 OPRS 的 5 个假设中的 3 个（H1，H4 和 H5），适用于住院医师的自我评估。
与通用评估工具 GOALS 相比，OPRS 作为一种特定于程序的评估工具，能够更好地区分预期具有不同手术表现水平的组别。鉴于对基于视频的学习的兴趣，需要进一步开发有效的特定于程序的工具，以支持住院医师在一系列程序中进行基于视频的自我评估。
