Man Kaiwen, Harring Jeffrey R, Jiao Hong, Zhan Peida
University of Maryland, College Park, USA.
Authors share the first authorship.
Appl Psychol Meas. 2019 Nov;43(8):639-654. doi: 10.1177/0146621618824853. Epub 2019 Feb 22.
Computer-based testing (CBT) is becoming increasingly popular in assessing test-takers' latent abilities and making inferences regarding their cognitive processes. In addition to collecting item responses, an important benefit of using CBT is that response times (RTs) can also be recorded and used in subsequent analyses. To better understand the structural relations between multidimensional cognitive attributes and the working speed of test-takers, this research proposes a joint-modeling approach that integrates compensatory multidimensional latent traits and response speediness using item responses and RTs. The joint model is cast as a multilevel model in which the structural relation between working speed and accuracy are connected through their variance-covariance structures. The feasibility of this modeling approach is investigated via a Monte Carlo simulation study using a Bayesian estimation scheme. The results indicate that integrating RTs increased model parameter recovery and precision. In addition, Program of International Student Assessment (PISA) 2015 mathematics standard unit items are analyzed to further evaluate the feasibility of the approach to recover model parameters.
基于计算机的测试(CBT)在评估考生的潜在能力以及对其认知过程进行推断方面正变得越来越流行。除了收集题目作答情况外,使用CBT的一个重要好处是还可以记录反应时间(RTs)并用于后续分析。为了更好地理解多维认知属性与考生工作速度之间的结构关系,本研究提出了一种联合建模方法,该方法使用题目作答情况和反应时间来整合补偿性多维潜在特质和反应速度。联合模型被构建为一个多层次模型,其中工作速度和准确性之间的结构关系通过它们的方差协方差结构相连。使用贝叶斯估计方案通过蒙特卡罗模拟研究来研究这种建模方法的可行性。结果表明,整合反应时间提高了模型参数恢复和精度。此外,对2015年国际学生评估项目(PISA)数学标准单元题目进行分析,以进一步评估该方法恢复模型参数的可行性。