Lucas Brian P, D'Addio Antonietta, Block Clay, Manning Harold L, Remillard Brian, Leiter James C
Medicine Service, White River Junction VA Medical Center, White River Junction, VT, USA.
Geisel School of Medicine at Dartmouth College, Hanover, NH, USA.
Ultrasound J. 2019 Feb 22;11(1):4. doi: 10.1186/s13089-019-0119-6.
Current methods of assessing competence in acquiring point-of-care ultrasound images are inadequate. They rely upon cumbersome rating systems that do not depend on the actual outcome measured and lack evidence of validity. We describe a new method that uses a rigorous statistical model to assess performance of individual trainees based on the actual task, image acquisition. Measurements obtained from the images acquired (the actual desired outcome) are themselves used to validate effective training and competence acquiring ultrasound images. We enrolled a convenience sample of 21 spontaneously breathing adults from a general medicine ward. In random order, two trainees (A and B) and an instructor contemporaneously acquired point-of-care ultrasound images of the inferior vena cava and the right internal jugular vein from the same patients. Blinded diameter measurements from each ultrasound were analyzed quantitatively using a multilevel model. Consistent mean differences between each trainee's and the instructor's images were ascribed to systematic acquisition errors, indicative of poor measurement technique and a need for further training. Wider variances were attributed to sporadic errors, indicative of inconsistent application of measurement technique across patients. In addition, the instructor recorded qualitative observations of each trainee's performance during image acquisition.
For all four diameters, the means and variances of measurements from trainee A's images differed significantly from the instructor's, whereas those from trainee B's images were comparable. Techniques directly observed by the instructor supported these model-derived findings. For example, mean anteroposterior diameters of the internal jugular vein obtained from trainee A's images were 3.8 mm (90% CI 2.3-5.4) smaller than from the instructor's; this model-derived finding matched the instructor's observation that trainee A compressed the vein during acquisition. Instructor summative assessments agreed with model-derived findings, providing internal validation of the descriptive and quantitative assessments of competence acquiring ultrasound images.
Clinical measurements obtained from point-of-care ultrasound images acquired contemporaneously by trainees and an instructor can be used to quantitatively assess the image acquisition competence of specific trainees. This method may obviate resource-intensive qualitative rating systems that are based on ultrasound image quality and direct observation, while also helping instructors guide remediation.
目前评估获取床旁超声图像能力的方法并不完善。这些方法依赖于繁琐的评分系统，该系统不依赖于所测量的实际结果，且缺乏有效性证据。我们描述了一种新方法，该方法使用严格的统计模型，根据实际任务（图像采集）来评估个体受训者的表现。从采集的图像中获得的测量值（实际期望的结果）本身用于验证有效的培训以及获取超声图像的能力。我们从普通内科病房选取了21名自主呼吸的成年人作为便利样本。两名受训者（A和B）和一名指导教师以随机顺序同时从同一患者获取下腔静脉和右颈内静脉的床旁超声图像。使用多级模型对每个超声图像的盲法直径测量值进行定量分析。将每个受训者与指导教师图像之间一致的平均差异归因于系统性采集误差，这表明测量技术不佳且需要进一步培训。更大的方差归因于偶发性误差，这表明在不同患者中测量技术的应用不一致。此外，指导教师在图像采集过程中记录了对每个受训者表现的定性观察。
对于所有四个直径，受训者A图像测量值的均值和方差与指导教师的有显著差异，而受训者B图像的测量值与指导教师的相当。指导教师直接观察到的技术支持了这些模型得出的结果。例如，从受训者A图像获得的颈内静脉前后径均值比指导教师的小3.8毫米（90%置信区间2.3 - 5.4）；这个模型得出的结果与指导教师观察到的受训者A在采集过程中压迫静脉的情况相符。指导教师的总结性评估与模型得出的结果一致，为获取超声图像能力的描述性和定量评估提供了内部验证。
由受训者和指导教师同时采集的床旁超声图像获得的临床测量值可用于定量评估特定受训者的图像采集能力。这种方法可能避免基于超声图像质量和直接观察的资源密集型定性评分系统，同时也有助于指导教师进行补救指导。
