Vatral Caleb, Biswas Gautam, Cohn Clayton, Davalos Eduardo, Mohammed Naveeduddin
Open Ended Learning Environments, Department of Computer Science, Institute for Software Integrated Systems, Vanderbilt University, Nashville, TN, United States.
Front Artif Intell. 2022 Jul 22;5:941825. doi: 10.3389/frai.2022.941825. eCollection 2022.
Simulation-based training (SBT) programs are commonly employed by organizations to train individuals and teams for effective workplace cognitive and psychomotor skills in a broad range of applications. Distributed cognition has become a popular cognitive framework for the design and evaluation of these SBT environments, with structured methodologies such as used for analysis. However, the analysis and evaluations generated by such distributed cognition frameworks require extensive domain-knowledge and manual coding and interpretation, and the analysis is primarily qualitative. In this work, we propose and develop the application of multimodal learning analysis techniques to SBT scenarios. Using these analysis methods, we can use the rich multimodal data collected in SBT environments to generate more automated interpretations of trainee performance that supplement and extend traditional DiCoT analysis. To demonstrate the use of these methods, we present a case study of nurses training in a mixed-reality manikin-based (MRMB) training environment. We show how the combined analysis of the video, speech, and eye-tracking data collected as the nurses train in the MRMB environment supports and enhances traditional qualitative DiCoT analysis. By applying such quantitative data-driven analysis methods, we can better analyze trainee activities online in SBT and MRMB environments. With continued development, these analysis methods could be used to provide targeted feedback to learners, a detailed review of training performance to the instructors, and data-driven evidence for improving the environment to simulation designers.
基于模拟的培训（SBT）项目通常被组织用于培训个人和团队，使其在广泛的应用场景中具备有效的职场认知和心理运动技能。分布式认知已成为设计和评估这些SBT环境的一种流行认知框架，其中使用了诸如[具体结构化方法]等结构化方法进行分析。然而，这种分布式认知框架所产生的分析和评估需要广泛的领域知识以及人工编码和解读，并且分析主要是定性的。在这项工作中，我们提出并开发了多模态学习分析技术在SBT场景中的应用。使用这些分析方法，我们可以利用在SBT环境中收集的丰富多模态数据，对学员表现生成更自动化的解读，以补充和扩展传统的分布式认知任务分析（DiCoT）。为了演示这些方法的使用，我们展示了一个在基于混合现实人体模型（MRMB）的培训环境中对护士进行培训的案例研究。我们展示了在护士于MRMB环境中培训时收集的视频、语音和眼动追踪数据的综合分析如何支持并增强传统的定性DiCoT分析。通过应用这种基于定量数据的分析方法，我们可以在SBT和MRMB环境中更好地在线分析学员活动。随着不断发展，这些分析方法可用于向学习者提供有针对性的反馈，向教师提供对培训表现的详细评估，并为模拟设计师改进环境提供数据驱动的证据。
