Zackoff Matthew, Jerardi Karen, Unaka Ndidi, Sucharew Heidi, Klein Melissa
Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio.
Hosp Pediatr. 2015 Jun;5(6):342-7. doi: 10.1542/hpeds.2014-0134.
Residents play a critical role in the education of peers and medical students, yet attainment of teaching skills is not routinely assessed. The primary aim of this study was to develop a novel, skill-based Observed Structured Teaching Evaluation (OSTE) and self-assessment survey to measure the impact of a resident-as-teacher curriculum on teaching competency. The secondary aim was to determine interrater reliability of the OSTE.
A prospective study quantitatively assessed intern teaching competency via videotaped teaching encounters (videos) before and after a month-long hospital medicine rotation and self-assessment surveys over a 5-month period. The intervention group received the resident-as-teacher curriculum. Videos were evaluated by 2 blinded faculty via an OSTE covering 9 skills within 3 core components: preparation, teaching, and reflection. Pre- to post-HM rotation month differences were evaluated within and between groups using the Wilcoxon signed rank test and Wilcoxon rank-sum test, respectively.
Twenty-two of 25 (88%) control and 27 of 28 (96%) intervention interns participated; 100% of participants completed the study. The intervention group's pre-post difference for the total OSTE score and the average self-assessed competence statistically improved; however, no significant difference was seen between groups. The difference in preparation scores was significant for the intervention compared with the control. The OSTE's interrater reliability demonstrated good agreement with weighted kappas of 0.86 for preparation, 0.71 for teaching, and 0.93 for reflection.
Implementation of an objective, skill-based OSTE detected observable changes in interns' teaching competency after implementation of a brief resident-as-teacher curriculum. The OSTE's good interrater reliability may allow standardized assessment of skill attainment over time.
住院医师在对同行和医学生的教育中发挥着关键作用，但教学技能的获得情况却未得到常规评估。本研究的主要目的是开发一种新颖的、基于技能的观察性结构化教学评估（OSTE）和自我评估调查，以衡量住院医师作为教师的课程对教学能力的影响。次要目的是确定OSTE的评分者间信度。
一项前瞻性研究通过为期一个月的医院内科轮转前后的录像教学环节（视频）以及为期5个月的自我评估调查，对实习医生的教学能力进行了定量评估。干预组接受了住院医师作为教师的课程。由2名不知情的教员通过涵盖3个核心要素（准备、教学和反思）中的9项技能的OSTE对视频进行评估。使用Wilcoxon符号秩检验和Wilcoxon秩和检验分别评估组内和组间住院医内科轮转前后的差异。
25名对照组实习生中的22名（88%）和28名干预组实习生中的27名（96%）参与了研究；100%的参与者完成了研究。干预组的OSTE总分和平均自我评估能力的前后差异在统计学上有改善；然而，两组之间没有显著差异。与对照组相比，干预组在准备分数上的差异显著。OSTE的评分者间信度显示出良好的一致性，准备方面的加权kappa值为0.86，教学方面为0.71，反思方面为0.93。
实施基于技能的客观OSTE发现，在实施简短的住院医师作为教师的课程后实习医生的教学能力有明显变化。OSTE良好的评分者间信度可能使随着时间推移对技能获得情况进行标准化评估成为可能。
