Rekman Janelle, Hamstra Stanley J, Dudek Nancy, Wood Timothy, Seabrook Christine, Gofton Wade
Department of Surgical Education, The University of Ottawa, Ottawa, Ontario, Canada.
Milestones Research and Evaluation at the Accreditation Council for Graduate Medical Education, Chicago, Illinois.
J Surg Educ. 2016 Jul-Aug;73(4):575-82. doi: 10.1016/j.jsurg.2016.02.003. Epub 2016 Apr 1.
The shift toward competency-based medical education has created a demand for feasible workplace-based assessment tools. Perhaps, more important than competence to assess an individual patient is the ability to successfully manage a surgical clinic. Trainee performance in clinic is a critical component of learning to manage a surgical practice, yet no assessment tool currently exists to assess daily performance in outpatient clinics for surgery residents. The development of a competency-based assessment tool, the Ottawa Clinic Assessment Tool (OCAT), is described here to address this gap.
A consensus group of experts was gathered to generate dimensions of performance reflective of a competent "generalist" surgeon in clinic. A 6-month pilot study of the OCAT was conducted in orthopedics, general surgery, and obstetrics and gynecology with quantitative and qualitative evidence of validity collected. In all, 2 subsequent feedback sessions and a survey for staff and residents evaluated the OCAT for clarity and utility.
The OCAT is a 9-item tool, with a global assessment item and 2 short-answer questions. Among the 2 divisions, 44 staff surgeons completed 132 OCAT assessments of 79 residents. Psychometric data was collected as evidence of validity. Analysis of feedback indicated that the entrustability rating scale was useful for surgeons and residents and that the items could be correlated with individual competencies.
Multiple sources of validity evidence collected in this study demonstrate that the OCAT can measure resident clinic competency in a valid and feasible manner.
向基于胜任力的医学教育的转变产生了对可行的基于工作场所的评估工具的需求。或许，成功管理外科诊所的能力比评估单个患者的能力更为重要。实习生在诊所的表现是学习管理外科实践的关键组成部分，但目前尚无评估工具可用于评估外科住院医师在门诊诊所的日常表现。本文描述了一种基于胜任力的评估工具——渥太华诊所评估工具（OCAT）的开发，以填补这一空白。
召集了一个专家共识小组，以确定反映诊所中胜任的“全科”外科医生的表现维度。在骨科、普通外科以及妇产科对OCAT进行了为期6个月的试点研究，收集了有效性的定量和定性证据。总共进行了2次后续反馈会议以及一项针对工作人员和住院医师的调查，以评估OCAT的清晰度和实用性。
OCAT是一个包含9个项目的工具，有一个整体评估项目和2个简答题。在这两个科室中，44名外科主治医师完成了对79名住院医师的132次OCAT评估。收集了心理测量数据作为有效性的证据。反馈分析表明，可托付性评级量表对外科医生和住院医师有用，且这些项目可与个体胜任力相关联。
本研究收集的多种有效性证据来源表明，OCAT能够以有效且可行的方式测量住院医师在诊所的胜任力。
