Academic Unit of Medical Education, University of Sheffield, Sheffield, UK.
Br J Surg. 2011 Jul;98(7):1010-20. doi: 10.1002/bjs.7478. Epub 2011 Apr 8.
Most surgical assessment has been aimed at technical proficiency. However, non-technical skills also affect patient safety and clinical effectiveness. The NOTSS (Non-Technical Skills for Surgeons) assessment instrument was developed specifically to assess the non-technical skills of individual surgeons in the operating theatre. This study evaluated NOTSS as a real-world assessment, with a mix of minimally trained assessors. The evaluation criteria were feasibility, validity and psychometric reliability.
In a standard evaluation of NOTSS, 56 anaesthetists, 39 scrub nurses, two surgical care practitioners and three independent assessors provided 715 assessments of 404 surgical cases of 15 index procedures across six specialties performed by 85 surgical trainees.
The assessment was feasible, but important implementation challenges were highlighted. Most respondents considered the method valid, but with reservations about assessing cognition. The factor structure of scores, and their positive relationships with other measures of experience and performance, supported validity. Trainees' non-technical skill scores were relatively procedure-independent and achieved good reliability (generalizability coefficient 0·8 or more) when six to eight assessors observed one case each.
Minimally trained assessors, who are typically present in operating theatres, were sufficiently discriminating and consistent in their judgements of trainee surgeons' non-technical skills to provide reliable scores based on an achievable number of observations.
大多数外科评估都旨在提高技术水平。然而，非技术技能也会影响患者安全和临床效果。NOTSS（外科医生非技术技能）评估工具是专门为评估手术室中外科医生的非技术技能而开发的。本研究评估了 NOTSS 在现实世界中的应用，其中包括混合了一些非专业评估员。评估标准是可行性、有效性和心理测量可靠性。
在 NOTSS 的标准评估中，56 名麻醉师、39 名洗手护士、两名外科护理医师和三名独立评估员对 6 个专业的 15 项索引手术的 404 个手术病例中的 715 次评估进行了评估，这些手术由 85 名外科学员进行。
评估是可行的，但突出了一些重要的实施挑战。大多数受访者认为该方法有效，但对评估认知存在保留意见。评分的因素结构及其与其他经验和绩效衡量标准的积极关系支持了有效性。学员的非技术技能评分相对独立于手术程序，当六到八名评估员观察每个病例时，其可靠性（通用性系数为 0.8 或更高）良好。
在手术室中通常存在的经过最少培训的评估员在判断实习外科医生的非技术技能时具有足够的区分度和一致性，从而能够在可实现的观察次数的基础上提供可靠的评分。
