Haidari Tamim Ahmad, Bjerrum Flemming, Christensen Thomas Decker, Vad Henrik, Møller Lars Borgbjerg, Hansen Henrik Jessen, Konge Lars, Petersen René Horsleben
Department of Cardiothoracic Surgery, University Hospital of Copenhagen, Inge Lehmanns Vej 7, Section 2152, Rigshospitalet, Copenhagen, Denmark.
Department of Clinical Medicine, University of Copenhagen, Rigshospitalet, Copenhagen, Denmark.
Surg Endosc. 2022 Nov;36(11):8067-8075. doi: 10.1007/s00464-022-09235-5. Epub 2022 Apr 25.
To determine the number of procedures and expert raters necessary to provide a reliable assessment of competence in Video-Assisted Thoracoscopic Surgery (VATS) lobectomy.
Three randomly selected VATS lobectomies were performed on a virtual reality simulator by participants with varying experience in VATS. Video recordings of the procedures were independently rated by three blinded VATS experts using a modified VATS lobectomy assessment tool (VATSAT). The unitary framework of validity was used to describe validity evidence, and generalizability theory was used to explore the reliability of different assessment options.
Forty-one participants (22 novices, 10 intermediates, and 9 experienced) performed a total of 123 lobectomies. Internal consistency reliability, inter-rater reliability, and test-retest reliability were 0.94, 0.85, and 0.90, respectively. Generalizability theory found that a minimum of two procedures and four raters or three procedures and three raters were needed to ensure the overall reliability of 0.8. ANOVA showed significant differences in test scores between the three groups (P < 0.001). A pass/fail level of 19 out of 25 points was established using the contrasting groups' standard setting method, leaving one false positive (one novice passed) and zero false negatives (all experienced passed).
We demonstrated validity evidence for a VR simulator test with different lung lobes, and a credible pass/fail level was identified. Our results can be used to implement a standardized mastery learning training program for trainees in VATS lobectomies that ensures that everyone reaches basic competency before performing supervised operations on patients.
确定对电视辅助胸腔镜手术（VATS）肺叶切除术能力进行可靠评估所需的操作数量和专家评分者数量。
经验各异的VATS参与者在虚拟现实模拟器上进行了3例随机选择的VATS肺叶切除术。手术视频记录由3名不知情的VATS专家使用改良的VATS肺叶切除术评估工具（VATSAT）进行独立评分。采用效度的单一框架来描述效度证据，并运用概化理论探索不同评估选项的可靠性。
41名参与者（22名新手、10名中级水平者和9名经验丰富者）共进行了123例肺叶切除术。内部一致性信度、评分者间信度和重测信度分别为0.94、0.85和0.90。概化理论发现，至少需要2次操作和4名评分者或3次操作和3名评分者才能确保总体信度达到0.8。方差分析显示三组间测试分数存在显著差异（P < 0.001）。采用对比组标准设定方法确定了25分中19分的通过/失败水平，出现1例假阳性（1名新手通过）和0例假阴性（所有经验丰富者通过）。
我们证明了针对不同肺叶的VR模拟器测试的效度证据，并确定了可信的通过/失败水平。我们的结果可用于为VATS肺叶切除术学员实施标准化的掌握学习培训计划，确保每个人在对患者进行监督手术之前达到基本能力。
