Department of Gynecology and Obstetrics, Johns Hopkins University School of Medicine, Baltimore, Maryland; the Department of Obstetrics and Gynecology, Uniformed Services University of the Health Sciences, Bethesda, Maryland; the Department of Gynecology, Mayo Clinic, Jacksonville, Florida; the Department of Obstetrics and Gynecology, University of California Riverside, Riverside, California; the Department of Obstetrics and Gynecology, Oklahoma University College of Medicine, Oklahoma City, Oklahoma; Augmented Reality Systems Inc., Windham, New Hampshire; and the Department of Obstetrics and Gynecology, Wake Forest School of Medicine, Winston-Salem, North Carolina. At the time of study, all participating institutions were members of the American College of Obstetricians and Gynecologists Simulations Working Group.
Obstet Gynecol. 2020 Nov;136(5):942-949. doi: 10.1097/AOG.0000000000004085.
To use the Messick validity framework for a simulation-based assessment of vaginal hysterectomy skills.
Video recordings of physicians at different levels of training and experience performing vaginal hysterectomy on a high-fidelity vaginal surgery model were objectively assessed using a modified 10-item Vaginal Surgical Skills Index, a one-item global scale of overall performance, and a pass-fail criterion. Participants included obstetrics and gynecology trainees and faculty from five institutions. Video recordings were independently assessed by expert surgeons blinded to the identities of the study participants.
Fifty surgeons (11 faculty, 39 trainees) were assessed. Experience level correlated strongly with both the modified Vaginal Surgical Skills Index and global scale score, with more experienced participants receiving higher scores (Pearson r=0.81, P<.001; Pearson r=0.74, P<.001). Likewise, surgical experience was also moderately correlated with the modified Vaginal Surgical Skills Index and global scale score (Pearson r=0.55, P<.001; Pearson r=0.58, P<.001). The internal consistency of the modified Vaginal Surgical Skills Index was excellent (Cronbach's alpha=0.97). Interrater reliability of the modified Vaginal Surgical Skills Index and global scale score, as measured by the intraclass correlation coefficient, was moderate to good (0.49-0.95; 0.50-0.87). Using the receiver operating characteristic curve and the pass-fail criterion, a modified Vaginal Surgical Skills Index cutoff score of 27 was found to most accurately (area under the curve 0.951, 95% CI 0.917-0.983) differentiate competent from noncompetent surgeons.
We demonstrated validity evidence for using a high-fidelity vaginal surgery model with the modified Vaginal Surgical Skills Index or global scale score to assess vaginal hysterectomy skills.
使用 Messick 有效性框架对基于模拟的阴道子宫切除术技能进行评估。
使用改良的 10 项阴道手术技能指数、一项整体绩效综合评分和通过/不通过标准，对不同培训水平和经验的医生在高保真阴道手术模型上进行阴道子宫切除术的视频记录进行客观评估。参与者包括来自五所机构的妇产科住院医师和教员。视频记录由对研究参与者身份不知情的专家外科医生进行独立评估。
评估了 50 名外科医生（11 名教员，39 名学员）。经验水平与改良的阴道手术技能指数和整体评分高度相关，经验丰富的参与者得分更高（Pearson r=0.81，P<.001；Pearson r=0.74，P<.001）。同样，手术经验也与改良的阴道手术技能指数和整体评分中度相关（Pearson r=0.55，P<.001；Pearson r=0.58，P<.001）。改良的阴道手术技能指数的内部一致性极好（Cronbach's alpha=0.97）。改良的阴道手术技能指数和整体评分的组内相关系数（ICC）衡量的观察者间信度为中度至良好（0.49-0.95；0.50-0.87）。使用接受者操作特征曲线和通过/不通过标准，发现改良的阴道手术技能指数截断值为 27 时，最能准确区分有能力的和无能力的外科医生（曲线下面积 0.951，95%CI 0.917-0.983）。
我们证明了使用高保真阴道手术模型和改良的阴道手术技能指数或整体评分来评估阴道子宫切除术技能的有效性证据。
