Acosta Stefan, Sevonius Dan, Beckman Anders
Vascular Centre, Skåne University Hospital, Malmö, Sweden;
Department of Surgery, Skåne University Hospital, Malmö-Lund, Sweden.
Med Educ Online. 2015 Jan 29;20:25819. doi: 10.3402/meo.v20.25819. eCollection 2015.
The Basic Surgical Skills course uses an assessment score interval of 0-3. An extended score interval, 1-6, was proposed by the Swedish steering committee of the course. The aim of this study was to analyze the trainee scores in the current 0-3 scored version compared to a proposed 1-6 scored version.
Sixteen participants, seven females and nine males, were evaluated in the current and proposed assessment forms by instructors, observers, and learners themselves during the first and second day. In each assessment form, 17 tasks were assessed. The inter-rater reliability between the current and the proposed score sheets were evaluated with intraclass correlation (ICC) with 95% confidence intervals (CI).
The distribution of scores for 'knot tying' at the last time point and 'bowel anastomosis side to side' given by the instructors in the current assessment form showed that the highest score was given in 31 and 62%, respectively. No ceiling effects were found in the proposed assessment form. The overall ICC between the current and proposed score sheets after assessment by the instructors increased from 0.38 (95% CI 0.77-0.78) on Day 1 to 0.83 (95% CI 0.51-0.94) on Day 2.
A clear ceiling effect of scores was demonstrated in the current assessment form, questioning its validity. The proposed score sheet provides more accurate scores and seems to be a better feedback instrument for learning technical surgical skills in the Basic Surgical Skills course.
基础外科技能课程使用的评估分数区间为0 - 3分。该课程的瑞典指导委员会提议采用1 - 6分的扩展分数区间。本研究的目的是分析当前0 - 3分计分版本与提议的1 - 6分计分版本中学员的分数情况。
16名参与者，7名女性和9名男性，在第一天和第二天由教员、观察员以及学员自己根据当前和提议的评估表进行评估。在每种评估表中，对17项任务进行评估。使用组内相关系数（ICC）及95%置信区间（CI）评估当前和提议计分表之间的评分者间信度。
在当前评估表中，教员在最后一个时间点给出的“打结”和“端端肠吻合术”的分数分布显示，分别有31%和62%的情况给出了最高分。在提议的评估表中未发现天花板效应。教员评估后，当前和提议计分表之间的总体ICC从第一天的0.38（95% CI 0.77 - 0.78）增加到第二天的0.83（95% CI 0.51 - 0.94）。
当前评估表中显示出明显的分数天花板效应，对其有效性提出了质疑。提议的计分表能提供更准确的分数，似乎是基础外科技能课程中学习外科技术技能的更好反馈工具。
