Liu Yue, Li Zhen, Liu Hongyun, Luo Fang
Beijing Key Laboratory of Applied Experimental Psychology, Faculty of Psychology, Beijing Normal University, Beijing, China.
eMetric LLC, San Antonio, TX, United States.
Front Psychol. 2019 Feb 4;10:145. doi: 10.3389/fpsyg.2019.00145. eCollection 2019.
The validity of inferences based on test scores will be threatened when examinees' test-taking non-effort is ignored. A possible solution is to add test-taking effort indicators in the measurement model after the non-effortful responses are flagged. As a new application of the multidimensional item response theory (MIRT) model for non-ignorable missing responses, this article proposed a MIRT method to account for non-effortful responses. Two simulation studies were conducted to examine the impact of non-effortful responses on item and latent ability parameter estimates, and to evaluate the performance of the MIRT method, comparing to the three-parameter logistic (3PL) model as well as the effort-moderated model. Results showed that: (a) as the percentage of non-effortful responses increased, the unidimensional 3PL model yielded poorer parameter estimates; (b) the MIRT model could obtain as accurate item parameter estimates as the effort-moderated model; (c) the MIRT model provided the most accurate ability parameter estimates when the correlation between test-taking effort and ability was high. A real data analysis was also conducted for illustration. The limitation and future research were discussed further.
当忽略考生的考试努力程度时，基于考试分数的推断的有效性将受到威胁。一种可能的解决方案是在标记出不认真作答后，在测量模型中加入考试努力程度指标。作为多维项目反应理论（MIRT）模型在不可忽略的缺失反应方面的新应用，本文提出了一种考虑不认真作答的MIRT方法。进行了两项模拟研究，以检验不认真作答对项目和潜在能力参数估计的影响，并评估MIRT方法的性能，与三参数逻辑斯蒂（3PL）模型以及努力程度调节模型进行比较。结果表明：（a）随着不认真作答的比例增加，单维3PL模型的参数估计效果变差；（b）MIRT模型能够获得与努力程度调节模型一样准确的项目参数估计；（c）当考试努力程度与能力之间的相关性较高时，MIRT模型提供的能力参数估计最为准确。还进行了实际数据分析以作说明。进一步讨论了局限性和未来研究方向。
