Watkins Scott C, Roberts David A, Boulet John R, McEvoy Matthew D, Weinger Matthew B
From the Department of Anesthesiology (S.CW., D.A.R., M.D.M., M.B.W.), Vanderbilt University School of Medicine; Center for Research and Innovation in Systems Safety (S.C.W., M.B.W.), Vanderbilt University Medical Center, Nashville, TN; and Foundation for Advancement of International Medical Education and Research (FAMER) (J.R.B), and the Educational Commission for Foreign Medical Graduates (ECFMG), Philadelphia, PA.
Simul Healthc. 2017 Apr;12(2):69-75. doi: 10.1097/SIH.0000000000000199.
Management of critical events requires teams to employ nontechnical skills (NTS), such as teamwork, communication, decision making, and vigilance. We sought to estimate the reliability and provide evidence for the validity of the ratings gathered using a new tool for assessing the NTS of anesthesia providers, the behaviorally anchored rating scale (BARS), and compare its scores with those of an established NTS tool, the Anaesthetists' Nontechnical Skills (ANTS) scale.
Six previously trained raters (4 novices and 2 experts) reviewed and scored 18 recorded simulated pediatric crisis management scenarios using a modified ANTS and a BARS tool. Pearson correlation coefficients were calculated separately for the novice and expert raters, by scenario, and overall.
The intrarater reliability of the ANTS total score was 0.73 (expert, 0.57; novice, 0.84); for the BARS tool, it was 0.80 (expert, 0.79; novice, 0.81). The average interrater reliability of BARS scores (0.58) was better than ANTS scores (0.37), and the interrater reliabilities of scores from novices (0.69 BARS and 0.52 ANTS) were better than those obtained from experts (0.47 BARS and 0.21 ANTS) for both scoring instruments. The Pearson correlation between the ANTS and BARS total scores was 0.74.
Overall, reliability estimates were better for the BARS scores than the ANTS scores. For both measures, the intrarater and interrater reliability was better for novices compared with domain experts, suggesting that properly trained novices can reliably assess the NTS of anesthesia providers managing a simulated critical event. There was substantial correlation between the 2 scoring instruments, suggesting that the tools measured similar constructs. The BARS tool can be an alternative to the ANTS scale for the formative assessment of NTS of anesthesia providers.
危急事件的管理要求团队运用非技术技能（NTS），如团队协作、沟通、决策和警觉性。我们试图评估使用一种用于评估麻醉医护人员非技术技能的新工具——行为锚定评级量表（BARS）所收集评级的可靠性，并为其有效性提供证据，同时将其得分与一种既定的非技术技能工具——麻醉医师非技术技能（ANTS）量表的得分进行比较。
六名先前经过培训的评分者（4名新手和2名专家）使用改良的ANTS和BARS工具对18个录制的模拟儿科危机管理场景进行审查和评分。分别按场景和总体计算新手和专家评分者的Pearson相关系数。
ANTS总分的评分者内信度为0.73（专家为0.57；新手为0.84）；对于BARS工具，其为0.80（专家为0.79；新手为0.81）。BARS评分的平均评分者间信度（0.58）优于ANTS评分（0.37），并且对于两种评分工具，新手评分的评分者间信度（BARS为0.69，ANTS为0.52）优于专家评分（BARS为0.47，ANTS为0.21）。ANTS和BARS总分之间的Pearson相关系数为0.74。
总体而言，BARS评分的可靠性评估优于ANTS评分。对于这两种测量方法，新手的评分者内和评分者间信度均优于领域专家，这表明经过适当培训后的新手能够可靠地评估处理模拟危急事件的麻醉医护人员的非技术技能。两种评分工具之间存在显著相关性，表明这两种工具测量的是相似的结构。BARS工具可作为ANTS量表的替代工具，用于麻醉医护人员非技术技能的形成性评估。
