Goff Barbara, Mandel Lynn, Lentz Gretchen, Vanblaricom Amy, Oelschlager Anne-Marie Amies, Lee David, Galakatos Andrew, Davies Matthew, Nielsen Peter
Division of Gynecologic Oncology, University of Washington School of Medicine, Seattle, WA 98195, USA.
Am J Obstet Gynecol. 2005 Apr;192(4):1331-8; discussion 1338-40. doi: 10.1016/j.ajog.2004.12.068.
We have previously shown that in a single residency program objective structured assessment of technical skills (OSATS) is a reliable and valid method of assessing surgical competency. Our goal was to establish feasibility of this evaluation instrument when administered at multiple residency programs throughout the US, and assess the impact of a laboratory-based surgical curriculum on results.
An OSATS exam was administered to 116 residents from 5 residency programs. One of the residency programs had participated in a comprehensive surgical curriculum over a 4-year period of time. The exam consisted of 3 open and 3 laparoscopic tasks. Residents were graded by both a blinded and unblinded examiner with task-specific checklist, global rating scale, pass/fail assessment, and tasks were timed. All tasks were performed on life-like models.
Examinations were successfully completed at all sites. Each exam required 90 minutes of resident time. Reliability indices calculated with Cronbach's alpha were .97 for overall global rating and .95 for checklists. Interrater reliability between blinded and unblinded examiners ranged from .71 to .97 for individual tasks and was .95 overall. Assessment of construct validity (the ability to distinguish among residency levels) found significant differences among the residents for both blinded and unblinded examiners for all evaluation outcomes except time. For the test overall, the global rating scale showed significant differences among all 4 residency levels. The checklist showed significant differences at three levels (PGY3-4 >PGY2 >PGY1). Approximate cost for replaceable items was 40 US dollars to 150 per resident depending on which tasks were chosen. Comparison of scores between residents who received a laboratory-based curriculum and those who did not revealed significantly higher scores and shorter time to complete tasks for the group who received additional training.
Large-scale testing has confirmed that OSATS is an objective, reliable, and valid method to assess surgical skills, and can easily be administered in most residency programs. A laboratory-based surgical curriculum improved test results and reduced time to complete tasks.
我们之前已经表明，在单一住院医师培训项目中，客观结构化技术技能评估（OSATS）是评估手术能力的一种可靠且有效的方法。我们的目标是确定在美国多个住院医师培训项目中实施该评估工具的可行性，并评估基于实验室的外科技能课程对评估结果的影响。
对来自5个住院医师培训项目的116名住院医师进行了OSATS考试。其中一个住院医师培训项目在4年期间参与了一项全面的外科技能课程培训。该考试包括3项开放性任务和3项腹腔镜任务。由一名盲法考官和一名非盲法考官根据特定任务清单、整体评分量表、通过/失败评估对住院医师进行评分，并对任务进行计时。所有任务均在仿真模型上进行。
所有地点均成功完成了考试。每次考试需要住院医师90分钟时间。用Cronbach's alpha计算的可靠性指标显示，整体整体评分的可靠性为0.97，清单的可靠性为0.95。盲法考官和非盲法考官之间的评分者间信度在各个任务中为0.71至0.97，总体为0.95。对结构效度（区分住院医师水平的能力）的评估发现，除时间外，对于所有评估结果，盲法考官和非盲法考官评估的住院医师之间均存在显著差异。对于整个测试，整体评分量表在所有4个住院医师水平之间均显示出显著差异。清单在三个水平上显示出显著差异（PGY3 - 4 > PGY2 > PGY1）。根据所选任务的不同，每位住院医师可替换物品的大致成本为40美元至150美元。接受基于实验室课程培训的住院医师与未接受培训的住院医师的分数比较显示，接受额外培训的组得分显著更高，完成任务的时间更短。
大规模测试证实，OSATS是一种客观、可靠且有效的手术技能评估方法，并且可以在大多数住院医师培训项目中轻松实施。基于实验室的外科技能课程提高了测试结果并缩短了完成任务的时间。
