Department of Pediatrics, Yale University School of Medicine, 333 Cedar Street, New Haven, CT, 06510, USA.
Department of Pediatrics, University of Washington School of Medicine, Seattle, USA.
BMC Med Educ. 2022 Aug 26;22(1):645. doi: 10.1186/s12909-022-03700-4.
To compare validity evidence for dichotomous and trichotomous versions of a neonatal intubation (NI) procedural skills checklist.
NI skills checklists were developed utilizing an existing framework. Experts were trained on scoring using dichotomous and trichotomous checklists, and rated recordings of 23 providers performing simulated NI. Videolaryngoscope recordings of glottic exposure were evaluated using Cormack-Lehane (CL) and Percent of Glottic Opening scales. Internal consistency and reliability of both checklists were analyzed, and correlations between checklist scores, airway visualization, entrustable professional activities (EPA), and global skills assessment (GSA) were calculated.
During rater training, raters gave significantly higher scores on better provider performance in standardized videos (both p < 0.001). When utilized to evaluate study participants' simulated NI attempts, both dichotomous and trichotomous checklist scores demonstrated very good internal consistency (Cronbach's alpha 0.868 and 0.840, respectively). Inter-rater reliability was higher for dichotomous than trichotomous checklists [Fleiss kappa of 0.642 and 0.576, respectively (p < 0.001)]. Sum checklist scores were significantly different among providers in different disciplines (p < 0.001, dichotomous and trichotomous). Sum dichotomous checklist scores correlated more strongly than trichotomous scores with GSA and CL grades. Sum dichotomous and trichotomous checklist scores correlated similarly well with EPA.
Neither dichotomous or trichotomous checklist was superior in discriminating provider NI skill when compared to GSA, EPA, or airway visualization assessment. Sum scores from dichotomous checklists may provide sufficient information to assess procedural competence, but trichotomous checklists may permit more granular feedback to learners and educators. The checklist selected may vary with assessment needs.
比较新生儿插管（NI）程序技能检查表的二分法和三分法版本的有效性证据。
使用现有框架开发了 NI 技能检查表。专家接受了使用二分法和三分法检查表进行评分的培训，并对 23 名进行模拟 NI 的提供者的录音进行了评分。使用 Cormack-Lehane（CL）和 Glottic Opening 比例评估喉镜记录的声门暴露情况。分析了两种检查表的内部一致性和可靠性，并计算了检查表评分、气道可视化、可委托的专业活动（EPA）和全球技能评估（GSA）之间的相关性。
在评分员培训期间，评分员在标准化视频中对表现更好的提供者给予了显著更高的评分（两者均 p<0.001）。当用于评估研究参与者的模拟 NI 尝试时，二分法和三分法检查表评分均表现出非常好的内部一致性（Cronbach 的 alpha 分别为 0.868 和 0.840）。二分法检查表的组内可靠性高于三分法检查表[Fleiss kappa 分别为 0.642 和 0.576（p<0.001）]。不同学科的提供者之间的总分检查表评分存在显著差异（p<0.001，二分法和三分法）。总分二分法检查表评分与 GSA 和 CL 等级的相关性强于三分法评分。总分二分法和三分法检查表评分与 EPA 的相关性相似。
与 GSA、EPA 或气道可视化评估相比，二分法或三分法检查表在区分提供者的 NI 技能方面都没有优势。来自二分法检查表的总分可能提供足够的信息来评估程序能力，但三分法检查表可能为学习者和教育者提供更细致的反馈。所选检查表可能因评估需求而异。
