University of Michigan Medical School, Ann Arbor, MI, USA.
Department of Learning Health Sciences, University of Michigan, Ann Arbor, MI, USA.
Surg Endosc. 2023 Sep;37(9):7170-7177. doi: 10.1007/s00464-023-10182-y. Epub 2023 Jun 19.
Laparoscopic training remains inaccessible for surgeons in low- and middle-income countries, limiting its widespread adoption. We developed a novel tool for assessment of laparoscopic appendectomy skills through ALL-SAFE, a low-cost laparoscopy training system.
This pilot study in Ethiopia, Cameroon, and the USA assessed appendectomy skills using the ALL-SAFE training system. Performance measures were captured using the ALL-SAFE verification of proficiency tool (APPY-VOP), consisting of a checklist, modified Objective Structured Assessment of Technical Skills (m-OSATS), and final rating. Twenty participants, including novice (n = 11), intermediate (n = 8), and expert (n = 1), completed an online module covering appendicitis management and psychomotor skills in laparoscopic appendectomy. After viewing an expert skills demonstration video, participants recorded their performance within ALL-SAFE. Using the APPY-VOP, participants rated their own and three peer videos. We used the Kruskal-Wallis test and a Many-Facet Rasch Model to evaluate (i) capacity of APPY-VOP to differentiate performance levels, (ii) correlation among three APPY-VOP components, and (iii) rating differences across groups.
Checklist scores increased from novice (M = 21.02) to intermediate (M = 23.64) and expert (M = 28.25), with differentiation between experts and novices, P = 0.005. All five m-OSATS domains and global summed, total summed, and final rating discriminated across all performance levels (P < 0.001). APPY-VOP final ratings adequately discriminated Competent (M = 2.0), Borderline (N = 1.8), and Not Competent (M = 1.4) performances, Χ (2,85) = 32.3, P = 0.001. There was a positive correlation between ALL-SAFE checklist and m-OSATS summed scores, r(83) = 0.63, P < 0.001. Comparison of ratings suggested no differences across expertise levels (P = 0.69) or location (P = 0.66).
APPY-VOP effectively discriminated between novice and expert performance in laparoscopic appendectomy skills in a simulated setting. Scoring alignment across raters suggests consistent evaluation, independent of expertise. These results support the use of APPY-VOP among all skill levels inside a peer rating system. Future studies will focus on correlating proficiency to clinical practice and scaling ALL-SAFE to other settings.
腹腔镜培训对于中低收入国家的外科医生来说仍然难以获得，这限制了其广泛采用。我们通过 ALL-SAFE，一种低成本的腹腔镜培训系统，开发了一种评估腹腔镜阑尾切除术技能的新工具。
这项在埃塞俄比亚、喀麦隆和美国进行的试点研究使用 ALL-SAFE 培训系统评估了阑尾切除术技能。使用 ALL-SAFE 熟练度验证工具（APPY-VOP）捕获性能指标，该工具包括检查表、改良客观结构化手术技能评估（m-OSATS）和最终评分。20 名参与者，包括新手（n=11）、中级（n=8）和专家（n=1），完成了涵盖阑尾炎管理和腹腔镜阑尾切除术心理运动技能的在线模块。在观看了专家技能示范视频后，参与者在 ALL-SAFE 中记录了他们的表现。使用 APPY-VOP，参与者对自己和三个同伴的视频进行评分。我们使用 Kruskal-Wallis 检验和多方面 Rasch 模型评估了以下内容：(i)APPY-VOP 区分绩效水平的能力；(ii)三个 APPY-VOP 组成部分之间的相关性；(iii)不同组别之间的评分差异。
检查表评分从新手（M=21.02）增加到中级（M=23.64）和专家（M=28.25），专家与新手之间存在差异，P=0.005。所有五个 m-OSATS 领域和全球总和、总总和、最终评分均在所有绩效水平上具有区分度（P<0.001）。APPY-VOP 最终评分能够充分区分熟练（M=2.0）、边缘（N=1.8）和不熟练（M=1.4）表现，Χ(2,85)=32.3，P=0.001。ALL-SAFE 检查表和 m-OSATS 总和评分之间存在正相关，r(83)=0.63，P<0.001。评分比较表明，在专业水平（P=0.69）或地点（P=0.66）上没有差异。
APPY-VOP 在模拟环境中有效地区分了腹腔镜阑尾切除术技能的新手和专家表现。评分在评分者之间的一致性表明，评估是一致的，与专业水平无关。这些结果支持在同伴评分系统中在所有技能水平上使用 APPY-VOP。未来的研究将集中在将熟练程度与临床实践相关联，并将 ALL-SAFE 扩展到其他环境。
