Cirigliano Matthew M, Guthrie Charles D, Pusic Martin V
Steinhardt School of Education, New York University, New York, New York, USA.
Graduate School of Arts and Sciences, New York University, New York, New York, USA.
Teach Learn Med. 2020 Aug-Sep;32(4):410-421. doi: 10.1080/10401334.2020.1754216. Epub 2020 May 12.
Learning in digital environments allows the collection of inexpensive, fine-grained process data across a large population of learners. Intentional design of the data collection can enable iterative testing of an instructional design. In this study, we propose that across a population of learners the information from multiple choice question responses can help to identify which design features are associated with positive learner engagement. We hypothesized that, within an online module that presents serial knowledge content, measures of click-level behavior will show sufficient, but variable, association with a test-measure so as to potentially guide instructional design. The Aquifer online learning platform employs interactive approaches to enable effective learning of health professions content. A multidisciplinary focus group of experts identified potential learning analytic measures within an Aquifer learning module, including: hyperlinks clicked (yes/no), magnify buttons clicked (yes/no), expert advice links clicked (yes/no), and time spent on each page (seconds). Learning analytics approaches revealed which click-level data was correlated with the subsequent relevant Case MCQ. We report regression coefficients where the dependent variable is student accuracy on the Case MCQ as a general indicator of successful engagement. Clicking hyperlinks, magnifying images, clicking "expert" links, and spending >100 seconds on each page were learning analytic measures and were positively correlated with Case MCQ success; rushing through pages (<20 seconds) was inversely correlated with success. Conversely, for some measures, we failed to find expected associations. In online learning environments, the wealth of process data available offers insights for instructional designers to iteratively hone the effectiveness of learning. Learning analytic measures of engagement can provide feedback as to which interaction elements are effective.
在数字环境中学习能够收集大量学习者的低成本、细粒度过程数据。对数据收集进行有意设计可以对教学设计进行迭代测试。在本研究中，我们提出，在众多学习者中，来自多项选择题答案的信息有助于确定哪些设计特征与积极的学习者参与度相关。我们假设，在呈现系列知识内容的在线模块中，点击级行为的测量将显示出与测试测量有足够但可变的关联，从而有可能指导教学设计。Aquifer在线学习平台采用交互式方法来促进健康专业内容的有效学习。一个多学科专家焦点小组确定了Aquifer学习模块中的潜在学习分析测量指标，包括：是否点击超链接、是否点击放大按钮、是否点击专家建议链接以及在每个页面上花费的时间（秒）。学习分析方法揭示了哪些点击级数据与后续相关的病例多项选择题相关。我们报告了回归系数，其中因变量是学生在病例多项选择题上的准确率，作为成功参与度的一般指标。点击超链接、放大图像、点击“专家”链接以及在每个页面上花费超过100秒是学习分析测量指标，与病例多项选择题的成功呈正相关；快速浏览页面（<20秒）与成功呈负相关。相反，对于一些测量指标，我们未能找到预期的关联。在在线学习环境中，可用的大量过程数据为教学设计人员提供了见解，以便他们迭代地优化学习效果。参与度的学习分析测量指标可以提供关于哪些交互元素有效的反馈。
