Huang Hung-Yu
University of Taipei, Taiwan.
Educ Psychol Meas. 2020 Dec;80(6):1168-1195. doi: 10.1177/0013164420914711. Epub 2020 Apr 24.
In educational assessments and achievement tests, test developers and administrators commonly assume that test-takers attempt all test items with full effort and leave no blank responses with unplanned missing values. However, aberrant response behavior-such as performance decline, dropping out beyond a certain point, and skipping certain items over the course of the test-is inevitable, especially for low-stakes assessments and speeded tests due to low motivation and time limits, respectively. In this study, test-takers are classified as normal or aberrant using a mixture item response theory (IRT) modeling approach, and aberrant response behavior is described and modeled using item response trees (IRTrees). Simulations are conducted to evaluate the efficiency and quality of the new class of mixture IRTree model using WinBUGS with Bayesian estimation. The results show that the parameter recovery is satisfactory for the proposed mixture IRTree model and that treating missing values as ignorable or incorrect and ignoring possible performance decline results in biased estimation. Finally, the applicability of the new model is illustrated by means of an empirical example based on the Program for International Student Assessment.
在教育评估和成绩测试中,测试开发者和管理者通常假定考生会全力以赴完成所有测试项目,不会留下无计划缺失值的空白答案。然而,异常作答行为——如成绩下降、在某个时间点后退出以及在测试过程中跳过某些题目——是不可避免的,尤其是对于低风险评估和限时测试,分别是由于动机不足和时间限制。在本研究中,使用混合项目反应理论(IRT)建模方法将考生分类为正常或异常,并使用项目反应树(IRTrees)描述和建模异常作答行为。使用WinBUGS进行贝叶斯估计的模拟,以评估新的混合IRT树模型的效率和质量。结果表明,所提出的混合IRT树模型的参数恢复情况令人满意,将缺失值视为可忽略或错误处理以及忽略可能的成绩下降会导致有偏差的估计。最后,通过基于国际学生评估项目的实证例子说明了新模型的适用性。