Leventhal Brian C, Pastor Dena
James Madison University, Harrisonburg, VA, USA.
Educ Psychol Meas. 2024 Aug;84(4):810-834. doi: 10.1177/00131644231185533. Epub 2023 Jul 26.
Low-stakes test performance commonly reflects examinee ability and effort. Examinees exhibiting low effort may be identified through rapid guessing behavior throughout an assessment. There has been a plethora of methods proposed to adjust scores once rapid guesses have been identified, but these have been plagued by strong assumptions or the removal of examinees. In this study, we illustrate how an IRTree model can be used to adjust examinee ability for rapid guessing behavior. Our approach is flexible as it does not assume independence between rapid guessing behavior and the trait of interest (e.g., ability) nor does it necessitate the removal of examinees who engage in rapid guessing. In addition, our method uniquely allows for the simultaneous modeling of a disengagement latent trait in addition to the trait of interest. The results indicate the model is quite useful for estimating individual differences among examinees in the disengagement latent trait and in providing more precise measurement of examinee ability relative to models ignoring rapid guesses or accommodating it in different ways. A simulation study reveals that our model results in less biased estimates of the trait of interest for individuals with rapid responses, regardless of sample size and rapid response rate in the sample. We conclude with a discussion of extensions of the model and directions for future research.
低风险测试成绩通常反映考生的能力和努力程度。在整个评估过程中，表现出低努力程度的考生可能会通过快速猜测行为被识别出来。一旦识别出快速猜测行为，已经提出了大量方法来调整分数，但这些方法一直受到强假设或剔除考生的困扰。在本研究中，我们说明了如何使用IRT树模型来针对快速猜测行为调整考生能力。我们的方法很灵活，因为它既不假设快速猜测行为与感兴趣的特质（如能力）之间存在独立性，也不要求剔除进行快速猜测的考生。此外，我们的方法独特地允许在对感兴趣的特质进行建模的同时，对脱离潜在特质进行建模。结果表明，该模型对于估计考生在脱离潜在特质方面的个体差异以及相对于忽略快速猜测或以不同方式适应快速猜测的模型而言，在提供更精确的考生能力测量方面非常有用。一项模拟研究表明，无论样本大小和样本中的快速反应率如何，我们的模型对于具有快速反应的个体在感兴趣特质的估计上偏差更小。我们最后讨论了模型的扩展以及未来研究的方向。
