Shayne Philip, Gallahue Fiona, Rinnert Stephan, Anderson Craig L, Hern Gene, Katz Eric
Department of Emergency Medicine, Emory University, Atlanta, GA 30305, USA.
Acad Emerg Med. 2006 Jul;13(7):727-32. doi: 10.1197/j.aem.2006.01.030. Epub 2006 Apr 24.
A Council of Emergency Medicine Residency Directors task force developed the Standardized Direct Observation Assessment Tool (SDOT), a 26-item checklist assessment tool to evaluate Accreditation Council for Graduate Medical Education resident core competencies by direct observation. Each of the checklist items is assigned to one or more of five core competencies. The objective of this study was to test the interrater measurement properties of the SDOT instrument.
Two videos of simulated patient-resident-attending physician encounters were produced. Academic emergency medicine faculty members not involved in the development of the form viewed the two encounters and completed the SDOT for each. Faculty demographic data were collected. Data were collected from 82 faculty members at 16 emergency medicine residency programs. The checklist items were used to generate a composite score for each core competency of patient care, medical knowledge, interpersonal and communication skills, professionalism, and systems-based practice.
Univariate analysis demonstrated a high degree of agreement between evaluators in evaluating residents for both videos. Multivariate analysis found no differences in rating by faculty when examined by experience, academic title, site, or previous use of the SDOT.
Faculty from 16 emergency medicine residency programs had a high interrater agreement when using the SDOT to evaluate resident core competency performance. This study did not test the validity of the tool. This data analysis is mainly descriptive, and scripted video scenarios may not approximate direct observation in the emergency department.
急诊医学住院医师培训主任委员会的一个特别工作组开发了标准化直接观察评估工具（SDOT），这是一种包含26个条目的清单式评估工具，用于通过直接观察来评估研究生医学教育认证委员会住院医师的核心能力。清单中的每个条目都被分配到五个核心能力中的一个或多个。本研究的目的是测试SDOT工具的评分者间测量属性。
制作了两段模拟患者 - 住院医师 - 主治医师会诊的视频。未参与该表格开发的急诊医学专业教员观看了这两段会诊视频，并为每段视频完成SDOT评估。收集了教员的人口统计学数据。从16个急诊医学住院医师培训项目的82名教员处收集了数据。使用清单条目为患者护理、医学知识、人际沟通技能、职业素养和基于系统的实践这五个核心能力中的每一个生成综合得分。
单变量分析表明，评估者在评估两段视频中的住院医师时具有高度一致性。多变量分析发现，按经验、学术职称、地点或之前是否使用过SDOT进行检查时，教员的评分没有差异。
来自16个急诊医学住院医师培训项目的教员在使用SDOT评估住院医师核心能力表现时，评分者间一致性较高。本研究未测试该工具的有效性。此数据分析主要是描述性的，且脚本化的视频场景可能无法近似急诊科的直接观察。
