Department of Obstetrics and Gynecology, Copenhagen University Hospital Amager and Hvidovre, Hvidovre, Denmark.
Department of Obstetrics and Gynecology, Roskilde University Hospital, Roskilde, Denmark.
Eur J Obstet Gynecol Reprod Biol. 2019 Jun;237:74-78. doi: 10.1016/j.ejogrb.2019.04.011. Epub 2019 Apr 15.
The aims of the study were to develop and gather validity evidence for a feasible rating scale for formative and summative assessment of total laparoscopic hysterectomy in the operating theatre.
The study was a prospective observer-blinded cohort study. The rating scale was developed according to the generic format of Objective Structured Assessment of Technical Skills. We applied the contemporary framework of validity to examine validity evidence of the content, response process, internal structure, relationship to other variables, and consequences. Two experienced gynecologists constructed a preliminary version of the rating scale, which was reviewed by a multicentre team of experienced gynecologists in a modified Delphi process. The surgeons (beginners and experienced surgeons) were video recorded during live performance of total laparoscopic hysterectomies. Two blinded raters evaluated the performances independently using the rating scale. Internal consistency reliability and interrater reliability were calculated as measures of internal structure. The performances of the two groups were compared and a pass/fail score was set to show the consequences of the rating scale.
The content of the rating scale was defined during three Delphi rounds and upon agreement comprised of 12 items. Sixteen participants including 8 beginners and 8 experienced surgeons performed total laparoscopic hysterectomies. The internal consistency reliability of the items was 0.95 (Cronbach's alpha), and the interrater reliabilities (Intraclass Correlation Coefficient, absolute agreement) were 0.996 for one rater and 0.998 for two raters (P < 0.001 for all correlations). The beginners' mean performance score was 19.2 (SD 7.1) and the experienced surgeons' score was 36.4 (SD 3.9); the groups performed statistically significantly different (P < 0.001). The pass/fail score was 29.3 with no false positives and no false negatives.
With this study, a feasible rating scale for the objective assessment of total laparoscopic hysterectomy was developed with sound validity evidence. The rating scale is suitable for both formative and summative feedback in the commencement of surgical training in gynecology.
本研究旨在开发并收集一种用于手术室腹腔镜全子宫切除术形成性和总结性评估的可行评分量表的效度证据。
本研究为前瞻性观察者盲法队列研究。评分量表根据客观结构化手术技能评估的通用格式制定。我们应用当代有效性框架来检查内容有效性证据、反应过程有效性证据、内部结构有效性证据、与其他变量的关系有效性证据和结果有效性证据。两位经验丰富的妇科医生构建了评分量表的初步版本，该版本由一个经验丰富的妇科医生多中心团队在修改后的 Delphi 过程中进行了审查。外科医生（初学者和经验丰富的外科医生）在腹腔镜全子宫切除术的现场演示中被录像。两名盲评员使用评分量表独立评估表现。内部一致性信度和评分者间信度作为内部结构的度量来计算。比较了两组的表现，并设置了及格/不及格分数来显示评分量表的结果。
评分量表的内容在三轮 Delphi 中进行了定义，经达成一致意见后由 12 个项目组成。16 名参与者，包括 8 名初学者和 8 名经验丰富的外科医生，进行了腹腔镜全子宫切除术。项目的内部一致性信度为 0.95（Cronbach's alpha），两名评分者的评分者间信度（组内相关系数，绝对一致）分别为 0.996 和 0.998（所有相关性的 P 值均<0.001）。初学者的平均表现评分为 19.2（SD 7.1），经验丰富的外科医生的评分为 36.4（SD 3.9）；两组的表现存在统计学显著差异（P<0.001）。及格/不及格分数为 29.3，无假阳性和假阴性。
通过这项研究，开发了一种用于腹腔镜全子宫切除术客观评估的可行评分量表，并具有良好的有效性证据。该评分量表适用于妇科手术培训开始时的形成性和总结性反馈。
