Dyre L, Nørgaard L N, Tabor A, Madsen M E, Sørensen J L, Ringsted C, Tolsgaard M
Juliane Marie Centre, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark.
Department of Gynaecology and Obstetrics, University of Copenhagen, Nordsjaelland Hospital, Hilleroed, Denmark.
Ultraschall Med. 2016 Aug;37(4):386-92. doi: 10.1055/s-0041-107976. Epub 2016 Mar 1.
To collect validity evidence for the assessment of mastery learning on a virtual reality transabdominal ultrasound simulator.
We assessed the validity evidence using Messick's framework for validity. The study included 20 novices and 9 ultrasound experts who all completed 10 obstetric training modules on a transabdominal ultrasound simulator that provided automated measures of performance for each completed module (i. e., simulator metrics). Differences in the performance of the two groups were used to identify simulator metrics with validity evidence for the assessment of mastery learning. The novices continued to practice until they had attained mastery learning level.
One-third of the simulator metrics discriminated between the two groups. The median simulator scores from a maximum of 40 metrics were 17.5 percent (range 0 - 45.0 percent) for novices and 90.0 percent (range 85.0 - 97.5) for experts, p < 0.001. Internal consistency was high, with a Cronbach's alpha value of 0.98. The test/retest reliability gave an intra-class correlation coefficient (ICC) of 0.62 for novices who reached the mastery learning level twice. Novices reached the mastery learning level within a median of 4 attempts (range 3 - 8) corresponding to a median of 252 minutes of simulator training (range 211 - 394 minutes).
This study found that validity evidence for the assessment of mastery learning in simulation-based ultrasound training can be demonstrated and that ultrasound novices can attain mastery learning levels with less than 5 hours of training. Only one-third of the standard simulator metrics discriminated between different levels of competence.
收集关于虚拟现实经腹超声模拟器上掌握学习评估的效度证据。
我们使用梅西克效度框架评估效度证据。该研究纳入了20名新手和9名超声专家，他们均在经腹超声模拟器上完成了10个产科训练模块，该模拟器为每个完成的模块提供性能自动测量值（即模拟器指标）。两组表现的差异用于识别具有效度证据的模拟器指标，以评估掌握学习情况。新手持续练习直至达到掌握学习水平。
三分之一的模拟器指标能够区分两组。从最多40个指标得出的模拟器得分中位数，新手为17.5%（范围0 - 45.0%），专家为90.0%（范围85.0 - 97.5），p < 0.001。内部一致性较高，克朗巴哈系数值为0.98。重测信度方面，两次达到掌握学习水平的新手组内相关系数（ICC）为0.62。新手达到掌握学习水平的尝试次数中位数为4次（范围3 - 8次），对应模拟器训练时间中位数为252分钟（范围211 - 394分钟）。
本研究发现，基于模拟的超声训练中掌握学习评估的效度证据可以得到证明，并且超声新手通过少于5小时的训练即可达到掌握学习水平。只有三分之一的标准模拟器指标能够区分不同能力水平。
