利用虚拟患者开发和初步验证在线参与度指标。
Development and initial validation of an online engagement metric using virtual patients.
机构信息
Dartmouth Geisel School of Medicine, One Rope Ferry Road, Hanover, NH, 03756, USA.
One Medical Center Drive, Bethesda, NH, 03756, Lebanon.
出版信息
BMC Med Educ. 2018 Sep 17;18(1):213. doi: 10.1186/s12909-018-1322-z.
BACKGROUND
Considerable evidence in the learning sciences demonstrates the importance of engagement in online learning environments. The purpose of this work was to demonstrate feasibility and to develop and collect initial validity evidence for a computer-generated dynamic engagement score based on student interactions in an online learning environment, in this case virtual patients used for clinical education.
METHODS
The study involved third-year medical students using virtual patient cases as a standard component of their educational program at more than 125 accredited US and Canadian medical schools. The engagement metric algorithm included four equally weighted components of student interactions with the virtual patient. We developed a self-report measure of motivational, emotional, and cognitive engagement and conducted confirmatory factor analysis to assess the validity of the survey responses. We gathered additional validity evidence through educator reviews, factor analysis of the metric, and correlations between student use of the engagement metric and self-report measures of learner engagement.
RESULTS
Confirmatory factor analysis substantiated the hypothesized four-factor structure of the survey scales. Educator reviews demonstrated a high level of agreement with content and scoring cut-points (mean Pearson correlation 0.98; mean intra-class correlation 0.98). Confirmatory factor analysis yielded an acceptable fit to a one-factor model of the engagement score components. Correlations of the engagement score with self-report measures were statistically significant and in the predicted directions.
CONCLUSIONS
We present initial validity evidence for a dynamic online engagement metric based on student interactions in a virtual patient case. We discuss potential uses of such an engagement metric including better understanding of student interactions with online learning, improving engagement through instructional design and interpretation of learning analytics output.
背景
学习科学领域有大量证据表明，参与在线学习环境非常重要。本研究旨在展示一种基于学生在在线学习环境（即用于临床教育的虚拟患者）中的互动而生成的计算机动态参与评分的可行性，并为其开发和收集初步有效性证据，该评分是通过学生与虚拟患者的互动来计算的。
方法
该研究涉及使用虚拟患者病例作为其教育计划标准组成部分的三年级医学生，这些学生来自 125 多所美国和加拿大认可的医学院。参与度指标算法包括学生与虚拟患者互动的四个同等权重的组成部分。我们开发了一种自我报告的动机、情感和认知参与度测量工具，并进行了验证性因素分析，以评估调查结果的有效性。我们通过教育工作者的评价、指标的因子分析以及学生使用参与度指标与自我报告的学习者参与度测量之间的相关性，收集了额外的有效性证据。
结果
验证性因素分析证实了调查量表假设的四因素结构。教育工作者的评价表明，内容和评分标准的一致性很高（平均皮尔逊相关系数为 0.98；平均组内相关系数为 0.98）。对参与度评分组成部分的验证性因素分析得出了一个可接受的单因素模型拟合度。参与度评分与自我报告测量的相关性具有统计学意义且符合预期方向。
结论
我们提出了一种基于虚拟患者病例中学生互动的动态在线参与度指标的初步有效性证据。我们讨论了这种参与度指标的潜在用途，包括更好地了解学生与在线学习的互动、通过教学设计和学习分析输出的解释来提高参与度。
Zotero 插件