Copenhagen Academy for Medical Education and Simulation, The Capital Region of Denmark, Copenhagen, Denmark.
Department of Vascular Surgery, Hospital Universitari de Bellvitge, Barcelona, Spain; European Society for Vascular Surgery (ESVS), France.
Eur J Vasc Endovasc Surg. 2020 Dec;60(6):933-941. doi: 10.1016/j.ejvs.2020.07.075. Epub 2020 Sep 6.
The aim of this study was to gather validity evidence for the Assessment of basic Vascular Ultrasound Expertise (AVAUSE) tool, and to establish a pass/fail score for each component, to support decisions for certification.
A cross sectional validation study performed during the European Society for Vascular Surgery's annual meeting. Validity evidence was sought for the theoretical test and two practical tests based on Messick's framework. The participants were vascular surgeons, vascular surgical trainees, sonographers, and nurses with varying experience levels. Five vascular ultrasound experts developed the theoretical and two practical test components of the AVAUSE tool for each test component. Two stations were set up for carotid examinations and two for superficial venous incompetence (SVI) examinations. Eight raters were assigned in pairs to each station. Three methods were used to set pass/fail scores: contrasting groups' method; rater consensus; and extended Angoff.
Nineteen participants were enrolled. Acceptable internal consistency reliability (Cronbach's alpha) for the AVAUSE theoretical (0.93), carotid (0.84), and SVI (0.65) practical test were shown. In the carotid examination, inter-rater reliability (IRR) for the two rater pairs was good: 0.68 and 0.78, respectively. The carotid scores correlated significantly with years of experience (Pearson's r = 0.56, p = .013) but not with number of examinations in the last five years. For SVI, IRR was excellent at 0.81 and 0.87. SVI performance scores did not correlate with years of experience and number of examinations. The pass/fail score set by the contrasting groups' method was 29 points out of 50. The rater set pass/fail scores were 3.0 points for both carotid and SVI examinations and were used to determine successful participants. Ten of 19 participants passed the tests and were certified.
Validity evidence was sought and established for the AVAUSE comprehensive tool, including pass/fail standards. AVAUSE can be used to assess competences in basic vascular ultrasound, allowing operators to progress towards independent practice.
本研究旨在收集评估基础血管超声专业技能（AVAUSE）工具的有效性证据，并为每个组成部分确定通过/不通过的分数，以支持认证决策。
这是一项在欧洲血管外科学会年会上进行的横断面验证研究。根据 Messick 框架，为理论测试和基于两个实践测试寻求有效性证据。参与者包括具有不同经验水平的血管外科医生、血管外科受训者、超声医师和护士。五位血管超声专家为每个测试组件开发了 AVAUSE 工具的理论和两个实践测试组件。为颈动脉检查设置了两个站，为浅静脉功能不全（SVI）检查设置了两个站。为每个站分配了两对八位评估员。使用三种方法设置通过/不通过分数：对比组方法；评估员共识；和扩展 Angoff 方法。
共纳入 19 名参与者。AVAUSE 理论（0.93）、颈动脉（0.84）和 SVI（0.65）实践测试的内部一致性可靠性（Cronbach's alpha）可接受。在颈动脉检查中，两个评估员对的组内信度（IRR）分别为 0.68 和 0.78，良好。颈动脉分数与年资显著相关（Pearson r=0.56，p=0.013），但与过去五年内的检查次数无关。对于 SVI，IRR 为 0.81 和 0.87，非常好。SVI 表现评分与年资和检查次数无关。通过对比组方法设置的通过/不通过分数为 50 分中的 29 分。评估员设定的通过/不通过分数分别为颈动脉和 SVI 检查的 3.0 分，用于确定通过的参与者。19 名参与者中有 10 名通过了测试并获得认证。
为 AVAUSE 综合工具寻求并建立了有效性证据，包括通过/不通过标准。AVAUSE 可用于评估基础血管超声的能力，使操作人员能够朝着独立实践的方向发展。
