Medi-StuNTS行为标记系统的有效性：在沉浸式模拟中评估医学生的非技术技能
Validity of the Medi-StuNTS behavioural marker system: assessing the non-technical skills of medical students during immersive simulation.
作者信息
Phillips Emma Claire, Smith Samantha Eve, Clarke Benjamin, Hamilton Ailsa Lauren, Kerins Joanne, Hofer Johanna, Tallentire Victoria Ruth
机构信息
Scottish Centre for Simulation and Clinical Human Factors, Larbert, UK.
NHS Lothian, Edinburgh, UK.
出版信息
BMJ Simul Technol Enhanc Learn. 2020 Apr 16;7(1):3-10. doi: 10.1136/bmjstel-2019-000506. eCollection 2021.
BACKGROUND
The Medical Students' Non-Technical Skills (Medi-StuNTS) behavioural marker system (BMS) is the first BMS to be developed specifically for medical students to facilitate training in non-technical skills (NTS) within immersive simulated acute care scenarios. In order to begin implementing the tool in practice, validity evidence must be sought. We aimed to assess the validity of the Medi-StuNTS system with reference to Messick's contemporary validity framework.
METHODS
Two raters marked video-recorded performances of acute care simulation scenarios using the Medi-StuNTS system. Three groups were marked: third-year and fourth-year medical students (novices), final-year medical students (intermediates) and core medical trainees (experts). The scores were used to make assessments of relationships to the variable of clinical experience through expert-novice comparisons, inter-rater reliability, observability, exploratory factor analysis, inter-rater disagreements and differential item functioning.
RESULTS
A significant difference was found between the three groups (p<0.005), with experts scoring significantly better than intermediates (p<0.005) and intermediates scoring significantly better than novices (p=0.001). There was a strong positive correlation between the two raters' scores (r=0.79), and an inter-rater disagreement of more than one point in less than one-fifth of cases. Across all scenarios, 99.7% of skill categories and 84% of skill elements were observable. Factor analysis demonstrated appropriate grouping of skill elements. Inconsistencies in test performance across learner groups were shown specifically in the skill categories of situation awareness and decision making and prioritisation.
CONCLUSION
We have demonstrated evidence for several aspects of validity of the Medi-StuNTS system when assessing medical students' NTS during immersive simulation. We can now begin to introduce this system into simulation-based education to maximise NTS training in this group.
背景
医学生非技术技能（Medi-StuNTS）行为标记系统（BMS）是首个专门为医学生开发的BMS，旨在促进在沉浸式模拟急性护理场景中进行非技术技能（NTS）培训。为了开始在实践中应用该工具，必须寻求效度证据。我们旨在参照梅西克的当代效度框架评估Medi-StuNTS系统的效度。
方法
两名评分者使用Medi-StuNTS系统对急性护理模拟场景的视频记录表现进行评分。共对三组进行评分：三年级和四年级医学生（新手）、五年级医学生（中级）和核心医学实习生（专家）。通过专家-新手比较、评分者间信度、可观察性、探索性因素分析、评分者间分歧和项目功能差异，利用这些分数对与临床经验变量的关系进行评估。
结果
三组之间存在显著差异（p<0.005），专家得分显著高于中级（p<0.005），中级得分显著高于新手（p=0.001）。两名评分者的分数之间存在很强的正相关（r=0.79），不到五分之一的案例中评分者间分歧超过一分。在所有场景中，99.7%的技能类别和84%的技能要素是可观察的。因素分析表明技能要素分组合理。不同学习者群体在测试表现上的不一致具体体现在态势感知、决策和优先级排序等技能类别中。
结论
我们已经证明了在沉浸式模拟中评估医学生NTS时Medi-StuNTS系统效度的几个方面。我们现在可以开始将这个系统引入基于模拟的教育中，以最大限度地提高该群体的NTS培训。
Zotero 插件