Nagy Gabriel, Ulitzsch Esther
Leibniz Institute for Science and Mathematics Education, Kiel, Germany.
Educ Psychol Meas. 2022 Oct;82(5):845-879. doi: 10.1177/00131644211045351. Epub 2021 Sep 13.
Disengaged item responses pose a threat to the validity of the results provided by large-scale assessments. Several procedures for identifying disengaged responses on the basis of observed response times have been suggested, and item response theory (IRT) models for response engagement have been proposed. We outline that response time-based procedures for classifying response engagement and IRT models for response engagement are based on common ideas, and we propose the distinction between independent and dependent latent class IRT models. In all IRT models considered, response engagement is represented by an item-level latent class variable, but the models assume that response times either reflect or predict engagement. We summarize existing IRT models that belong to each group and extend them to increase their flexibility. Furthermore, we propose a flexible multilevel mixture IRT framework in which all IRT models can be estimated by means of marginal maximum likelihood. The framework is based on the widespread Mplus software, thereby making the procedure accessible to a broad audience. The procedures are illustrated on the basis of publicly available large-scale data. Our results show that the different IRT models for response engagement provided slightly different adjustments of item parameters of individuals' proficiency estimates relative to a conventional IRT model.
未参与作答的项目反应对大规模评估结果的有效性构成威胁。已经提出了几种基于观察到的反应时间来识别未参与作答反应的程序，并且也提出了用于反应参与度的项目反应理论（IRT）模型。我们概述了基于反应时间的反应参与度分类程序和反应参与度的IRT模型是基于共同的理念，并且我们提出了独立和非独立潜在类别IRT模型之间的区别。在所有考虑的IRT模型中，反应参与度由项目层面的潜在类别变量表示，但这些模型假设反应时间要么反映要么预测参与度。我们总结了属于每个组的现有IRT模型并对其进行扩展以增加其灵活性。此外，我们提出了一个灵活的多级混合IRT框架，在该框架中所有IRT模型都可以通过边际极大似然法进行估计。该框架基于广泛使用的Mplus软件，从而使该程序可供广大受众使用。这些程序基于公开可用的大规模数据进行了说明。我们的结果表明，相对于传统IRT模型，不同的反应参与度IRT模型对个体能力估计的项目参数进行了略有不同的调整。
