Bell Colin, Wagner Natalie, Hall Andrew, Newbigging Joseph, Rang Louise, Vukadinovic Tatiana, McKaigney Conor
Department of Emergency Medicine and Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.
Department of Biomedical & Molecular Sciences, Queen's University, Kingston, ON, Canada.
CJEM. 2025 Feb;27(2):111-116. doi: 10.1007/s43678-024-00839-5. Epub 2025 Jan 3.
POCUS is a core emergency medicine skill and mainstay of early pregnancy assessment. The ultrasound competency assessment tool was developed as an entrustment-based assessment tool for use by content experts evaluating trainees performing multiple POCUS study types. The objective of this study was to evaluate the scoring and extrapolation inferences of the tool within Kane's validity framework when used to assess trainees performing an early pregnancy POCUS.
This was a multicentered study of emergency medicine residents participating in a POCUS assessment. After a background questionnaire, participants were read a case stem requesting a POCUS evaluation of an early pregnancy patient. Trainees were independently assessed by two fellowship-trained faculty. Descriptive statistics and two-way random, intraclass correlation coefficients, Cronbach's alpha were calculated on the merged data and used to assess all domains. Domain scores and an entrustment score for each participant were used to create a composite score. A one-way analysis of variance was performed.
36 trainees and 5 assessors completed the study. When used to assess trainee POCUS performance in early pregnancy, the tool demonstrated good to excellent interrater reliability for image acquisition, image generation, clinical integration, and entrustment (intraclass correlation coefficients 80-91 p < .001). The preparation domain had poor, but statistically significant interrater reliability (intraclass correlation coefficient 0.46 p = .04). An analysis of variance suggested the POCUS performance scores differed based on prior experience [F(2,32) = 3.74, p = .021).
This study adds further validity evidence relating to scoring and extrapolation of the ultrasound competency assessment tool when used to assess trainees performing a POCUS study in early pregnancy.
床旁超声（POCUS）是急诊医学的一项核心技能，也是早期妊娠评估的主要手段。超声能力评估工具是作为一种基于委托的评估工具而开发的，供内容专家用于评估进行多种POCUS研究类型的学员。本研究的目的是在Kane效度框架内评估该工具在用于评估进行早期妊娠POCUS的学员时的评分和外推推断。
这是一项对参与POCUS评估的急诊医学住院医师进行的多中心研究。在完成一份背景问卷后，向参与者宣读一个病例梗概，要求对一名早期妊娠患者进行POCUS评估。学员由两名接受过专科培训的教员独立评估。对合并后的数据计算描述性统计量、双向随机组内相关系数、Cronbach's α，并用于评估所有领域。使用每个参与者的领域分数和委托分数来创建一个综合分数。进行单向方差分析。
36名学员和5名评估者完成了研究。当用于评估学员在早期妊娠中的POCUS表现时，该工具在图像采集、图像生成、临床整合和委托方面表现出良好到优秀的评分者间信度（组内相关系数80 - 91，p < 0.001）。准备领域的评分者间信度较差，但具有统计学意义（组内相关系数0.46，p = 0.04）。方差分析表明，POCUS表现分数因先前经验而异[F(2,32) = 3.74，p = 0.021]。
本研究为超声能力评估工具在用于评估进行早期妊娠POCUS研究的学员时的评分和外推提供了进一步的效度证据。
