ITEC imec, Leuven, Belgium.
Faculty of Psychology and Educational Sciences, KU Leuven, Leuven, Belgium.
Behav Res Methods. 2019 Feb;51(1):332-341. doi: 10.3758/s13428-018-1129-1.
Massive open online courses (MOOCs) are increasingly popular among students of various ages and at universities around the world. The main aim of a MOOC is growth in students' proficiency. That is why students, professors, and universities are interested in the accurate measurement of growth. Traditional psychometric approaches based on item response theory (IRT) assume that a student's proficiency is constant over time, and therefore are not well suited for measuring growth. In this study we sought to go beyond this assumption, by (a) proposing to measure two components of growth in proficiency in MOOCs; (b) applying this idea in two dynamic extensions of the most common IRT model, the Rasch model; (c) illustrating these extensions through analyses of logged data from three MOOCs; and (d) checking the quality of the extensions using a cross-validation procedure. We found that proficiency grows both across whole courses and within learning objectives. In addition, our dynamic extensions fit the data better than does the original Rasch model, and both extensions performed well, with an average accuracy of .763 in predicting students' responses from real MOOCs.
大规模在线开放课程（MOOCs）在世界各地的学生和大学中越来越受欢迎。MOOC 的主要目的是提高学生的熟练程度。这就是为什么学生、教授和大学都对准确衡量增长感兴趣的原因。基于项目反应理论（IRT）的传统心理计量学方法假设学生的熟练程度在一段时间内是不变的，因此不太适合衡量增长。在这项研究中，我们试图超越这一假设，（a）提出在 MOOC 中衡量熟练程度增长的两个组成部分；（b）将这一想法应用于最常见的 IRT 模型——Rasch 模型的两个动态扩展；（c）通过对三个 MOOC 的记录数据进行分析来说明这些扩展；（d）使用交叉验证程序检查扩展的质量。我们发现，熟练程度不仅在整个课程中而且在学习目标内都在增长。此外，我们的动态扩展比原始的 Rasch 模型更适合数据，并且这两个扩展都表现良好，在预测真实 MOOC 中学生的反应方面的平均准确率为 0.763。
