Division of Gastroenterology and Hepatology, Medical University of South Carolina, Charleston, South Carolina, USA.
Division of Gastroenterology, Hepatology, and Nutrition, Learning Institute and Research Institute, Hospital for Sick Children, Toronto, Ontario, Canada.
Gastrointest Endosc. 2021 Apr;93(4):914-923. doi: 10.1016/j.gie.2020.07.055. Epub 2020 Jul 30.
The accurate measurement of technical skill in ERCP is essential for endoscopic training, quality assurance, and coaching of this procedure. Hypothesizing that technical skill can be measured by analysis of ERCP videos, we aimed to develop and validate a video-based ERCP skill assessment tool.
Based on review of procedural videos, the task of ERCP was deconstructed into its basic components by an expert panel that developed an initial version of the Bethesda ERCP Skill Assessment Tool (BESAT). Subsequently, 2 modified Delphi panels and 3 validation exercises were conducted with the goal of iteratively refining the tool. Fully crossed generalizability studies investigated the contributions of assessors, ERCP performance, and technical elements to reliability.
Twenty-nine technical elements were initially generated from task deconstruction. Ultimately, after iterative refinement, the tool comprised 6 technical elements and 11 subelements. The developmental process achieved consistent improvements in the performance characteristics of the tool with every iteration. For the most recent version of the tool, BESAT-v4, the generalizability coefficient (a reliability index) was .67. Most variance in BESAT scores (43.55%) was attributed to differences in endoscopists' skill, indicating that the tool can reliably differentiate between endoscopists based on video analysis.
Video-based assessment of ERCP skill appears to be feasible with a novel instrument that demonstrates favorable validity evidence. Future steps include determining whether the tool can discriminate between endoscopists of varying experience levels and predict important outcomes in clinical practice.
在 ERCP 中准确测量技术技能对于内镜培训、质量保证和该程序的指导至关重要。我们假设可以通过分析 ERCP 视频来衡量技术技能，因此旨在开发和验证一种基于视频的 ERCP 技能评估工具。
根据程序视频的审查，由专家小组将 ERCP 任务分解为其基本组成部分，该小组开发了贝塞斯达 ERCP 技能评估工具（BESAT）的初始版本。随后，进行了 2 次修改 Delphi 小组和 3 次验证练习，旨在迭代完善该工具。完全交叉的可概括性研究调查了评估者、ERCP 表现和技术要素对可靠性的贡献。
最初从任务解构中生成了 29 个技术要素。最终，经过反复改进，该工具包括 6 个技术要素和 11 个子要素。随着每一次迭代，工具的性能特征都取得了一致的改进。对于最新版本的工具，BESAT-v4 的可概括系数（可靠性指标）为.67。BESAT 评分的大部分差异（43.55%）归因于内镜医生技能的差异，这表明该工具可以通过视频分析可靠地区分内镜医生。
使用一种新的工具，基于视频的 ERCP 技能评估似乎是可行的，并且具有有利的有效性证据。未来的步骤包括确定该工具是否可以区分不同经验水平的内镜医生，以及预测临床实践中的重要结果。
