Department of Pediatrics (L Johnston,T Whitfill, C Bruno,O Levit, and M Auerbach), Yale University School of Medicine, New Haven, Conn.
Department of Pediatrics (T Sawyer), University of Washington School of Medicine, Seattle.
Acad Pediatr. 2019 Mar;19(2):157-164. doi: 10.1016/j.acap.2018.07.008. Epub 2018 Aug 11.
Neonatal tracheal intubation (NTI) is an important clinical skill. Suboptimal performance is associated with patient harm. Simulation training can improve NTI performance. Improving performance requires an objective assessment of competency. Competency assessment tools need strong evidence of validity. We hypothesized that an NTI competency assessment tool with multisource validity evidence could be developed and be used for formative and summative assessment during simulation-based training.
An NTI assessment tool was developed based on a literature review. The tool was refined through 2 rounds of a modified Delphi process involving 12 subject-matter experts. The final tool included a 22-item checklist, a global skills assessment, and an entrustable professional activity (EPA) level. The validity of the checklist was assessed by having 4 blinded reviewers score 23 videos of health care providers intubating a neonatal simulator.
The checklist items had good internal consistency (overall α = 0.79). Checklist scores were greater for providers at greater training levels and with more NTI experience. Checklist scores correlated with global skills assessment (ρ = 0.85; P < .05), EPA levels (ρ = 0.87; P < .05), percent glottic exposure (r = 0.59; P < .05), and Cormack-Lehane scores (ρ = 0.95; P < .05). Checklist scores reliably predicted EPA levels.
We developed an NTI competency assessment tool with multisource validity evidence. The tool was able to discriminate NTI performance based on experience. The tool can be used during simulation-based NTI training to provide formative and summative assessment and can aid with entrustment decisions.
新生儿气管插管（NTI）是一项重要的临床技能。操作不当与患者伤害有关。模拟训练可以提高 NTI 操作水平。要提高操作水平，就需要对能力进行客观评估。能力评估工具需要有强有力的有效性证据。我们假设，一种具有多源有效性证据的 NTI 能力评估工具可以被开发出来，并可用于基于模拟的培训中的形成性和总结性评估。
根据文献回顾开发了 NTI 评估工具。该工具通过涉及 12 名主题专家的 2 轮修改 Delphi 过程进行了改进。最终工具包括 22 项检查表、总体技能评估和可委托专业活动（EPA）水平。通过让 4 名盲审员对 23 名医护人员对新生儿模拟器进行插管的视频进行评分，评估了检查表的有效性。
检查表项目具有良好的内部一致性（总体 α=0.79）。在更高的培训水平和更多的 NTI 经验的提供者中，检查表的评分更高。检查表评分与总体技能评估（ρ=0.85；P<0.05）、EPA 水平（ρ=0.87；P<0.05）、声门暴露百分比（r=0.59；P<0.05）和 Cormack-Lehane 评分（ρ=0.95；P<0.05）相关。检查表评分可靠地预测了 EPA 水平。
我们开发了一种具有多源有效性证据的 NTI 能力评估工具。该工具能够根据经验区分 NTI 操作水平。该工具可用于基于模拟的 NTI 培训中提供形成性和总结性评估，并可辅助委托决策。
