Andreatta Pamela B, Klotz Jessica J, Dooley-Hash Suzanne L, Hauptman Joe G, Biddinger Bea, House Joseph B
University of Minnesota Medical School, Minneapolis (Dr Andreatta and Ms Klotz); University of Michigan Medical School, Ann Arbor (Drs Andreatta, Dooley-Hash, and House); Michigan State University College of Veterinary Medicine, East Lansing (Dr Hauptman and Ms Biddinger).
Adv Neonatal Care. 2015 Feb;15(1):56-64. doi: 10.1097/ANC.0000000000000130.
The purpose of this article was to establish psychometric validity evidence for competency assessment instruments and to evaluate the impact of 2 forms of training on the abilities of clinicians to perform neonatal intubation. To inform the development of assessment instruments, we conducted comprehensive task analyses including each performance domain associated with neonatal intubation. Expert review confirmed content validity. Construct validity was established using the instruments to differentiate between the intubation performance abilities of practitioners (N = 294) with variable experience (novice through expert). Training outcomes were evaluated using a quasi-experimental design to evaluate performance differences between 294 subjects randomly assigned to 1 of 2 training groups. The training intervention followed American Heart Association Pediatric Advanced Life Support and Neonatal Resuscitation Program protocols with hands-on practice using either (1) live feline or (2) simulated feline models. Performance assessment data were captured before and directly following the training. All data were analyzed using analysis of variance with repeated measures and statistical significance set at P < .05. Content validity, reliability, and consistency evidence were established for each assessment instrument. Construct validity for each assessment instrument was supported by significantly higher scores for subjects with greater levels of experience, as compared with those with less experience (P = .000). Overall, subjects performed significantly better in each assessment domain, following the training intervention (P = .000). After controlling for experience level, there were no significant differences among the cognitive, performance, and self-efficacy outcomes between clinicians trained with live animal model or simulator model. Analysis of retention scores showed that simulator trained subjects had significantly higher performance scores after 18 weeks (P = .01) and 52 weeks (P = .001) and cognitive scores after 52 weeks (P = .001). The results of this study demonstrate the feasibility of using valid, reliable assessment instruments to assess clinician competency and self-efficacy in the performance of neonatal intubation. We demonstrated the relative equivalency of live animal and simulation-based models as tools to support acquisition of neonatal intubation skills. Retention of performance abilities was greater for subjects trained using the simulator, likely because it afforded greater opportunity for repeated practice. Outcomes in each assessment area were influenced by the previous intubation experience of participants. This suggests that neonatal intubation training programs could be tailored to the level of provider experience to make efficient use of time and educational resources. Future research focusing on the uses of assessment in the applied clinical environment, as well as identification of optimal training cycles for performance retention, is merited.
本文旨在为能力评估工具建立心理测量效度证据，并评估两种培训形式对临床医生进行新生儿插管能力的影响。为指导评估工具的开发，我们进行了全面的任务分析，涵盖与新生儿插管相关的每个绩效领域。专家评审确认了内容效度。通过使用这些工具区分不同经验水平（从新手到专家）的从业者（N = 294）的插管表现能力，建立了结构效度。使用准实验设计评估培训结果，以评估随机分配到两个培训组之一的294名受试者之间的表现差异。培训干预遵循美国心脏协会儿科高级生命支持和新生儿复苏计划协议，采用实践操作，使用（1）活猫或（2）模拟猫模型。在培训前和培训刚结束后收集表现评估数据。所有数据均使用重复测量方差分析进行分析，设定统计学显著性为P < .05。为每个评估工具建立了内容效度、信度和一致性证据。与经验较少的受试者相比，经验更丰富的受试者在每个评估工具上的得分显著更高，支持了每个评估工具的结构效度（P = .000）。总体而言，在培训干预后，受试者在每个评估领域的表现都有显著提高（P = .000）。在控制经验水平后，使用活体动物模型或模拟器模型培训的临床医生在认知、表现和自我效能结果方面没有显著差异。留存分数分析表明，接受模拟器培训的受试者在18周（P = .01）和52周（P = .001）后的表现得分显著更高，在52周后的认知得分也显著更高（P = .001）。本研究结果证明了使用有效、可靠的评估工具来评估临床医生在新生儿插管操作中的能力和自我效能的可行性。我们证明了活体动物模型和基于模拟的模型作为支持获取新生儿插管技能的工具具有相对等效性。使用模拟器培训的受试者的表现能力留存率更高，可能是因为它提供了更多重复练习的机会。每个评估领域的结果都受到参与者之前插管经验的影响。这表明新生儿插管培训项目可以根据提供者的经验水平进行调整，以有效利用时间和教育资源。值得未来开展研究，重点关注评估在应用临床环境中的用途，以及确定表现留存的最佳培训周期。
