Department of Anaesthesia, Sir Charles Gairdner Hospital, Perth, WA 6009, Australia.
Br J Anaesth. 2010 Apr;104(4):440-5. doi: 10.1093/bja/aeq032. Epub 2010 Feb 26.
Performance assessment is becoming increasingly necessary in the medical workplace. Hospitals and patients expect safety, and under-performance by a doctor can compromise standards. By describing and quantifying performance, positive behaviour can be encouraged and unsafe behaviour remedied. Anaesthesia Non-Technical Skills (ANTS) is a behavioural marker system that can be used to assess non-technical skills in the workplace.
We determined whether specialist anaesthetists could be reliably trained to use ANTS at an assessor level in an 8 h programme. Unscripted videos of routine anaesthesia were produced for training and assessment purposes. Twenty-six participants attended rater training. Exercises in behaviour observation, rater error, frame of reference and performance dimension, and the use of ANTS were conducted throughout the day. Five videos were selected for formal assessment and data collected. Intra-class correlations (ICCs) were calculated for each element.
The accepted value of r>0.7 was not reached. ICC calculated for each element was 0.11-0.62. Comparison of participants scores with those of expert raters showed poor agreement.
Anaesthetists could not be trained to reliably use ANTS as a summative assessment tool using our 1 day programme. There was an inadequate correlation of scores between participants and experts. Two major problems contributed to the lack of agreement. Observed behaviours were often misclassified into the incorrect element and safety beliefs varied among anaesthetists. Other reasons for the failure to achieve success and potential future direction are discussed.
在医疗工作场所，绩效评估变得越来越必要。医院和患者都期望安全，如果医生表现不佳，可能会降低标准。通过描述和量化绩效，可以鼓励积极行为，纠正不安全行为。麻醉非技术技能（ANTS）是一种行为标记系统，可用于评估工作场所的非技术技能。
我们确定了经过 8 小时培训的专科麻醉师是否可以可靠地达到评估员水平，使用 ANTS。为培训和评估目的制作了常规麻醉的无脚本视频。26 名参与者参加了评估员培训。全天进行行为观察、评估者错误、参考框架和绩效维度以及 ANTS 使用方面的练习。选择了 5 个视频进行正式评估并收集数据。为每个元素计算了组内相关系数（ICC）。
未达到接受的 r>0.7 值。计算出的每个元素的 ICC 为 0.11-0.62。参与者得分与专家评分的比较显示出较差的一致性。
使用我们的 1 天方案，麻醉师无法接受培训以可靠地使用 ANTS 作为总结性评估工具。参与者和专家之间的评分相关性不足。有两个主要问题导致了缺乏一致性。观察到的行为经常被错误地归入不正确的元素，麻醉师的安全信念也存在差异。讨论了未能成功的其他原因和潜在的未来方向。
