Park Jung Yeon, Cornillie Frederik, van der Maas Han L J, Van Den Noortgate Wim
Faculty of Psychology and Educational Sciences, imec-ITEC, KU Leuven, Leuven, Belgium.
Department of Psychology, University of Amsterdam, Amsterdam, Netherlands.
Front Psychol. 2019 Mar 29;10:620. doi: 10.3389/fpsyg.2019.00620. eCollection 2019.
Adaptive learning systems have received an increasing attention as they enable to provide personalized instructions tailored to the behaviors and needs of individual learners. In order to reach this goal, it is desired to have an assessment system, monitoring each learner's ability change in real time. The Elo Rating System (ERS), a popular scoring algorithm for paired competitions, has recently been considered as a fast and flexible method that can assess learning progress in online learning environments. However, it has been argued that a standard ERS may be problematic due to the multidimensional nature of the abilities embedded in learning materials. In order to handle this issue, we propose a system that incorporates a multidimensional item response theory model (MIRT) in the ERS. The basic idea is that instead of updating a single ability parameter from the Rasch model, our method allows a simultaneous update of multiple ability parameters based on a compensatory MIRT model, resulting in a multidimensional extension of the ERS ("M-ERS"). To evaluate the approach, three simulation studies were conducted. Results suggest that the ERS that incorrectly assumes unidimensionality has a seriously lower prediction accuracy compared to the M-ERS. Accounting for both speed and accuracy in M-ERS is shown to perform better than using accuracy data only. An application further illustrates the method using real-life data from a popular educational platform for exercising math skills.
自适应学习系统越来越受到关注，因为它们能够提供根据个体学习者的行为和需求量身定制的个性化指导。为了实现这一目标，需要一个评估系统，实时监测每个学习者的能力变化。Elo评分系统（ERS）是一种用于配对比赛的流行评分算法，最近被认为是一种能够在在线学习环境中评估学习进度的快速且灵活的方法。然而，有人认为标准的ERS可能存在问题，因为学习材料中所包含的能力具有多维度性质。为了解决这个问题，我们提出了一种在ERS中纳入多维度项目反应理论模型（MIRT）的系统。其基本思想是，我们的方法不是从Rasch模型更新单个能力参数，而是基于补偿性MIRT模型同时更新多个能力参数，从而实现ERS的多维度扩展（“M-ERS”）。为了评估该方法，我们进行了三项模拟研究。结果表明，与M-ERS相比，错误地假设单维度性的ERS的预测准确性严重较低。研究表明，在M-ERS中同时考虑速度和准确性比仅使用准确性数据表现更好。一个应用实例进一步说明了该方法在使用来自一个流行的数学技能练习教育平台的实际数据时的情况。
