Department of Otolaryngology-Head and Neck Surgery, The Johns Hopkins University, School of Medicine, 601 N. Caroline Street, Baltimore, MD 21287, USA.
Am J Rhinol Allergy. 2010 May-Jun;24(3):234-7. doi: 10.2500/ajra.2010.24.3434. Epub 2009 Dec 16.
The Accreditation Council for Graduate Medical Education requires that residency programs have an effective plan for assessing their residents' competence. Current evaluation tools used for the assessment of operative competency are subjective, recall based, and do not allow providing formative feedback to the resident. The purpose of this study was to evaluate the feasibility and validity of video-based assessment of endoscopic sinus surgery (ESS) skills.
Five junior (postgraduate year levels 1-3[PGY-1 to -3]) and three senior (PGY-4 and -5) residents were videotaped as they performed ESS on cadavers in the laboratory. Five faculty members watched the recorded videotapes and rated performance using global and checklist assessment tools for ESS. Evaluators were blinded to the identity of the performers. Rating was made feasible by allowing the raters to fast-forward through the tapes when appropriate. Construct validity was calculated by comparing total scores and score on each item of the instrument across the training levels. For all statistical purposes p < 0.05 was considered significant.
Results showed construct validity with senior residents performing better than junior residents. It took an average of 20 minutes (range, 7- 39 minutes) to watch and evaluate a video. Interrater reliability, as measured by interclass correlation coefficient across evaluators, was 0.62.
The use of videotapes combined with the objective assessment tool is a feasible and valid method for evaluating surgical skills. It has implications for residency programs, because it will increase faculty buy-in and reduce the potential for biased assessment.
住院医师规范化培训评估委员会要求住院医师培训计划必须有一项有效的计划来评估住院医师的能力。目前用于评估手术能力的评估工具具有主观性、基于记忆，并且无法为住院医师提供形成性反馈。本研究旨在评估基于视频的鼻窦内窥镜手术（ESS）技能评估的可行性和有效性。
5 名初级（住院医师 1-3 年[PGY-1 至 -3]）和 3 名高级（PGY-4 和 -5）住院医师在实验室中对尸体进行 ESS 时被录像。5 名教员观看记录的录像，并使用 ESS 的整体和检查表评估工具对表现进行评分。评估者对表演者的身份一无所知。通过允许评估者在适当的时候快进录像，评分变得可行。通过比较各培训水平的总分和仪器各项目的得分来计算结构有效性。所有统计目的均认为 p < 0.05 具有统计学意义。
结果显示，高级住院医师的表现优于初级住院医师，具有结构有效性。观看和评估一段视频平均需要 20 分钟（范围 7-39 分钟）。评估者之间的组内相关系数衡量的组间可靠性为 0.62。
使用录像带结合客观评估工具是评估手术技能的一种可行且有效的方法。它对住院医师培训计划具有影响，因为它将增加教师的参与度，并降低评估的潜在偏差。
