Rogers-Vizena Carolyn R, Sideridis Georgios D, Patel Krishna G, Garland Catharine B, Mount Delora L, Yao Caroline A
Department of Plastic and Oral Surgery, Boston Children's Hospital, Boston, Mass.
Harvard Medical School, Boston, Mass.
Plast Reconstr Surg Glob Open. 2020 Jul 14;8(7):e2954. doi: 10.1097/GOX.0000000000002954. eCollection 2020 Jul.
Objective evaluation of operative performance is increasingly important in surgical training. Evaluation tools include global rating scales of performance and procedure-specific skills checklists. For unilateral cleft lip repair, the numerous techniques make universal evaluation challenging. Thus, we sought to create a unilateral cleft lip evaluation tool agnostic to specific repair technique.
Four surgeons with expertise in 3 common cleft lip repair techniques participated in a 3-round Delphi process to generate consensus evaluation points spanning all techniques. Items were categorized as marking the repair, performing the repair, and final result. Two blinded raters then scored videos of simulated cleft lip repairs using both the 21-item novel checklist and the modified Objective Structured Assessment of Technical Skills. Kappa and values were calculated for both scales to determine level of agreement.
Ten videos of repairs performed by novice residents through experienced craniofacial fellows were scored. Moderate (κ = 0.41-0.60) to substantial (κ = 0.61-0.80) interrater reliability was seen for the majority of questions in both the novel tool and the Objective Structured Assessment of Technical Skills. A single question in the novel tool had almost perfect agreement (κ = 0.81-1.00), 8 had moderate agreement, and 6 had substantial agreement. Poorly scoring questions were discarded from the final 18-item tool.
Despite variations in unilateral cleft lip repair technique, common themes exist that can be used to assess performance and outcome. A universal evaluation tool has potential implications for trainee assessment, surgeon credentialing, and screening for surgical missions.
在外科手术培训中，对手术操作进行客观评估变得越来越重要。评估工具包括整体绩效评分量表和特定手术技能清单。对于单侧唇裂修复，众多技术使得进行通用评估具有挑战性。因此，我们试图创建一种与特定修复技术无关的单侧唇裂评估工具。
四位精通三种常见唇裂修复技术的外科医生参与了三轮德尔菲法，以生成涵盖所有技术的共识评估要点。项目分为标记修复、进行修复和最终结果三类。然后，两名盲评者使用21项新清单和改良的客观结构化技术技能评估对模拟唇裂修复视频进行评分。计算两种量表的卡方值和相关数值以确定一致程度。
对新手住院医师至经验丰富的颅面外科研究员所进行的10次修复视频进行了评分。新工具和客观结构化技术技能评估中的大多数问题都显示出中等（κ = 0.41 - 0.60）至高度（κ = 0.61 - 0.80）的评分者间信度。新工具中的一个问题几乎完全一致（κ = 0.81 - 1.00），8个问题有中等一致性，6个问题有高度一致性。评分较低的问题从最终的18项工具中剔除。
尽管单侧唇裂修复技术存在差异，但仍存在可用于评估手术操作和结果的共同主题。一种通用评估工具对学员评估、外科医生资格认证和手术任务筛选可能具有潜在意义。
