McNally Jillian, Wilson David, Jones Emma E, Shahisavandi Neyousha, King Caroline, Coles Chad, Glennie Raymond Andrew
Division of Orthopaedics, Department of Surgery, Dalhousie University, Halifax, Nova Scotia, Canada.
Nova Scotia Health, Halifax Infirmary, Halifax, Nova Scotia Canada.
JB JS Open Access. 2025 Apr 25;10(2). doi: 10.2106/JBJS.OA.24.00183. eCollection 2025 Apr-Jun.
Surgical skills are critical to assess in residency programs. These observations often occur in the clinical settings, which are limited by patient safety and potential bias. High fidelity simulated cadaveric surgery can account for some of these shortcomings. Professional video offers a promising avenue to both anonymize and effectively evaluate surgical skill. The objective of this study were to describe the technique for professional video capture of simulated, open orthopaedic surgeries and to assess construct validity by comparing objective performance scores from the videos with the learner's stage of training.
In 2022, one experienced surgeon and 3 trainees (post graduate year [PGY]-4, PGY-3, PGY-2) were recruited from a residency program to perform 2 moderately challenging surgeries (open reduction and internal fixation of both bone forearm and talus fractures), with fractures simulated using an osteotome. Videographers positioned cameras at various positions throughout a skills laboratory. Total costs were calculated. Statistical analysis was performed to compare evaluator scores of participants' actual level of training.
The simulated surgeries were recorded, edited for optimal viewing angles, and anonymized by blurring faces and voice over technology. Seventeen local teaching faculty were recruited to evaluate the videos. The videos were shortened on average 65 minutes for critical steps to be represented in the final production (i.e., Bone reduction, dissection of neurovascular structures, radiographic images, etc.) The full cost to produce the 8 surgical videos was $48,934.00 Canadian dollars. The final data set had 61 observations, with a range of 13 to 17 observations per participant. There was a 19.7% error rate, meaning the videos were generally 80% accurate in predicting the year of training.
The discriminative ability of the videos was better at detecting true "novice" and "expert" surgeons but less accurate between the middle years of training. A larger, multicentered study with more participants is needed to draw any further conclusions.
在住院医师培训项目中，评估手术技能至关重要。这些观察通常发生在临床环境中，而临床环境受到患者安全和潜在偏差的限制。高保真模拟尸体手术可以弥补其中一些不足。专业视频为匿名化和有效评估手术技能提供了一条有前景的途径。本研究的目的是描述模拟开放性骨科手术的专业视频拍摄技术，并通过将视频中的客观表现评分与学习者的培训阶段进行比较来评估结构效度。
2022年，从一个住院医师培训项目中招募了一名经验丰富的外科医生和3名实习生（四年级住院医师[PGY]-4、三年级住院医师[PGY]-3、二年级住院医师[PGY]-2），进行2台中等难度的手术（双骨前臂骨折和距骨骨折的切开复位内固定），骨折用骨刀模拟。摄像师在技能实验室的不同位置放置摄像机。计算总成本。进行统计分析以比较评估者对参与者实际培训水平的评分。
模拟手术被录制下来，编辑以获得最佳视角，并通过面部模糊和语音覆盖技术进行匿名化处理。招募了17名当地教师来评估这些视频。为了在最终制作中展示关键步骤（即骨折复位、神经血管结构解剖、影像学图像等），视频平均缩短了65分钟。制作这8个手术视频的总成本为48,934.00加元。最终数据集有61条观察结果，每位参与者的观察结果范围为13至17条。错误率为19.7%，这意味着这些视频在预测培训年份方面总体准确率为80%。
这些视频在检测真正的“新手”和“专家”外科医生方面的辨别能力较好，但在培训中期之间的准确性较低。需要进行一项更大规模、多中心且有更多参与者的研究才能得出进一步的结论。
