参照系训练：建立对教学效果的可靠评估

Frame-of-Reference Training: Establishing Reliable Assessment of Teaching Effectiveness.

作者信息

Newman Lori R, Brodsky Dara, Jones Richard N, Schwartzstein Richard M, Atkins Katharyn Meredith, Roberts David H

机构信息

Ms. Newman: Principal Associate in Medicine, Harvard Medical School; Director of Professional Development in Medical Education, Department of Medical Education, Boston Children's Hospital, Boston, MA, and, at the time of this study, Director of Faculty Education, Carl J. Shapiro Institute for Education and Research at Harvard Medical School and Beth Israel Deaconess Medical Center, Boston, MA. Dr. Brodsky: Assistant Professor of Pediatrics, Department of Pediatrics, Harvard Medical School, and Associate Director, Neonatal Intensive Care Unit, Beth Israel Deaconess Medical Center, Boston, MA. Dr. Jones: Associate Professor of Psychiatry and Human Behavior, Warren Alpert Medical School of Brown University, Providence, RI. Dr. Schwartzstein: Professor of Medicine and Medical Education, Department of Medicine, Harvard Medical School, and Executive Director, Carl J. Shapiro Institute for Education and Research at Harvard Medical School and Beth Israel Deaconess Medical Center, and Vice President for Education, Beth Israel Deaconess Medical Center, Boston, MA. Dr. Atkins: Assistant Professor of Obstetrics, Gynecology, and Reproductive Biology, Harvard Medical School, and Director, Undergraduate Medical Education, Beth Israel Deaconess Medical Center, Boston, MA. Dr. Roberts: Associate Professor of Medicine, Department of Medicine, and Dean for External Education, Harvard Medical School, Boston, MA.

出版信息

J Contin Educ Health Prof. 2016 Summer;36(3):206-10. doi: 10.1097/CEH.0000000000000086.

DOI:10.1097/CEH.0000000000000086

PMID:27583997

Abstract

INTRODUCTION

Frame-of-reference (FOR) training has been used successfully to teach faculty how to produce accurate and reliable workplace-based ratings when assessing a performance. We engaged 21 Harvard Medical School faculty members in our pilot and implementation studies to determine the effectiveness of using FOR training to assess health professionals' teaching performances.

METHODS

All faculty were novices at rating their peers' teaching effectiveness. Before FOR training, we asked participants to evaluate a recorded lecture using a criterion-based peer assessment of medical lecturing instrument. At the start of training, we discussed the instrument and emphasized its precise behavioral standards. During training, participants practiced rating lectures and received immediate feedback on how well they categorized and scored performances as compared with expert-derived scores of the same lectures. At the conclusion of the training, we asked participants to rate a post-training recorded lecture to determine agreement with the experts' scores.

RESULTS

Participants and experts had greater rating agreement for the post-training lecture compared with the pretraining lecture. Through this investigation, we determined that FOR training is a feasible method to teach faculty how to accurately and reliably assess medical lectures.

DISCUSSION

Medical school instructors and continuing education presenters should have the opportunity to be observed and receive feedback from trained peer observers. Our results show that it is possible to use FOR rater training to teach peer observers how to accurately rate medical lectures. The process is time efficient and offers the prospect for assessment and feedback beyond traditional learner evaluation of instruction.

摘要

引言

参照系（FOR）培训已成功用于教导教员在评估绩效时如何做出准确可靠的基于工作场所的评分。我们让21位哈佛医学院教员参与了我们的试点和实施研究，以确定使用FOR培训来评估卫生专业人员教学表现的有效性。

方法

所有教员在对同行的教学效果进行评分方面都是新手。在进行FOR培训之前，我们要求参与者使用基于标准的医学讲座同伴评估工具对一段录制的讲座进行评估。在培训开始时，我们讨论了该工具并强调了其精确的行为标准。在培训过程中，参与者练习对讲座进行评分，并收到关于他们与同一讲座的专家评分相比，在对表现进行分类和评分方面做得如何的即时反馈。在培训结束时，我们要求参与者对培训后的一段录制讲座进行评分，以确定与专家评分的一致性。