Zargham Shiva, Hanson Amy, Laniewicz Megan, Sandquist Mary, Kessler David O, Gilbert Gregory E, Calhoun Aaron W
Department of Pediatrics University of Louisville School of Medicine Louisville KY USA.
and the Department of Emergency Medicine Columbia University Vagelos College of Physicians & Surgeons New York NY USA.
AEM Educ Train. 2020 Jun 17;5(2):e10482. doi: 10.1002/aet2.10482. eCollection 2021 Apr.
Effective trainee-led debriefing after critical events in the pediatric emergency department has potential to improve patient care, but debriefing assessments for this context have not been developed. This study gathers preliminary validity and reliability evidence for the Debriefing Assessment for Simulation in Healthcare (DASH) as an assessment of trainee-led post-critical event debriefing.
Eight fellows led teams in three simulated critical events, each followed by a video-recorded discussion of performance mimicking impromptu debriefings occurring after real clinical events. Three raters assessed the recorded debriefings using the DASH, and their feedback was collated. Data were analyzed using generalizability theory, Gwet's AC, intraclass correlation coefficient (ICC), and coefficient alpha. Validity was examined using Messick's framework.
The DASH instrument had relatively low traditional inter-rater reliability (Gwet's AC = 0.24, single-rater ICC range = 0.16-0.35), with 30% fellow, 19% rater, and 23% rater by fellow variance. DASH generalizability (G) coefficient was 0.72, confirming inadequate reliability for research purposes. Decision (D) study results suggest the DASH can attain a G coefficient of 0.8 with five or more raters. Coefficient alpha was 0.95 for the DASH. A total of 90 and 40% of items from Elements 1 and 4, respectively, were deemed "not applicable" or left blank.
Our results suggest that the DASH does not have sufficient validity and reliability to rigorously assess debriefing in the post-critical event environment but may be amenable to modification. Further development of the tool will be needed for optimal use in this context.
在儿科急诊科的危急事件后，由实习医生主导的有效病例汇报有改善患者护理的潜力，但针对这种情况的汇报评估尚未开发出来。本研究收集了医疗保健模拟病例汇报评估（DASH）作为对实习医生主导的危急事件后病例汇报评估的初步效度和信度证据。
八名研究员在三个模拟危急事件中带领团队，每个事件之后都有一段模仿真实临床事件后即兴病例汇报的表现视频讨论。三名评分者使用DASH对录制的病例汇报进行评估，并整理他们的反馈。使用概化理论、格韦特AC、组内相关系数（ICC）和阿尔法系数对数据进行分析。使用梅西克框架检验效度。
DASH工具的传统评分者间信度相对较低（格韦特AC = 0.24，单评分者ICC范围 = 0.16 - 0.35），研究员方差占30%，评分者方差占19%，研究员与评分者共同方差占23%。DASH概化（G）系数为0.72，证实其信度不足以用于研究目的。决策（D）研究结果表明，DASH在有五名或更多评分者时可达到0.8的G系数。DASH的阿尔法系数为0.95。分别有90%和40%的来自要素1和要素4的项目被视为“不适用”或留空。
我们的结果表明，DASH没有足够的效度和信度来严格评估危急事件后环境中的病例汇报，但可能适合进行修改。在此背景下，需要对该工具进行进一步开发以实现最佳使用。
