Sun Xiaojian, Liu Yanlou, Xin Tao, Song Naiqing
School of Mathematics and Statistics, Southwest University, Chongqing, China.
Southwest University Branch, Collaborative Innovation Center of Assessment for Basic Education Quality, Chongqing, China.
Front Psychol. 2020 Dec 2;11:575141. doi: 10.3389/fpsyg.2020.575141. eCollection 2020.
Calibration errors are inevitable and should not be ignored during the estimation of item parameters. Items with calibration error can affect the measurement results of tests. One of the purposes of the current study is to investigate the impacts of the calibration errors during the estimation of item parameters on the measurement accuracy, average test length, and test efficiency for variable-length cognitive diagnostic computerized adaptive testing. The other purpose is to examine the methods for reducing the adverse effects of calibration errors. Simulation results show that (1) calibration error has negative effect on the measurement accuracy for the deterministic input, noisy "and" gate (DINA) model, and the reduced reparameterized unified model; (2) the average test lengths is shorter, and the test efficiency is overestimated for items with calibration errors; (3) the compensatory reparameterized unified model (CRUM) is less affected by the calibration errors, and the classification accuracy, average test length, and test efficiency are slightly stable in the CRUM framework; (4) methods such as improving the quality of items, using large calibration sample to calibrate the parameters of items, as well as using cross-validation method can reduce the adverse effects of calibration errors on CD-CAT.
校准误差是不可避免的，在项目参数估计过程中不应被忽视。存在校准误差的项目会影响测试的测量结果。本研究的目的之一是调查项目参数估计过程中的校准误差对可变长度认知诊断计算机自适应测试的测量准确性、平均测试长度和测试效率的影响。另一个目的是研究减少校准误差不利影响的方法。模拟结果表明：（1）校准误差对确定性输入噪声“与”门（DINA）模型和简化重参数化统一模型的测量准确性有负面影响；（2）对于存在校准误差的项目，平均测试长度较短，且测试效率被高估；（3）补偿重参数化统一模型（CRUM）受校准误差的影响较小，在CRUM框架下分类准确性、平均测试长度和测试效率略有稳定；（4）提高项目质量、使用大校准样本校准项目参数以及使用交叉验证方法等可以减少校准误差对认知诊断计算机自适应测试（CD-CAT）的不利影响。
