Cheng Ying, Patton Jeffrey M, Shao Can
University of Notre Dame, Notre Dame, IN, USA.
Educ Psychol Meas. 2015 Apr;75(2):260-283. doi: 10.1177/0013164414530719. Epub 2014 Apr 21.
-Stratified computerized adaptive testing with -blocking (AST), as an alternative to the widely used maximum Fisher information (MFI) item selection method, can effectively balance item pool usage while providing accurate latent trait estimates in computerized adaptive testing (CAT). However, previous comparisons of these methods have treated item parameter estimates as if they are the true population parameter values. Consequently, capitalization on chance may occur. In this article, we examined the performance of the AST method under more realistic conditions where item parameter instead of true parameter values are used in the CAT. Its performance was compared against that of the MFI method when the latter is used in conjunction with Sympson-Hetter or randomesque exposure control. Results indicate that the MFI method, even when combined with exposure control, is susceptible to capitalization on chance. This is particularly true when the calibration sample size is small. On the other hand, AST is more robust to capitalization on chance. Consistent with previous investigations using true item parameter values, AST yields much more balanced item pool usage, with a small loss in the precision of latent trait estimates. The loss is negligible when the test is as long as 40 items.
带有分组的分层计算机自适应测试(AST),作为广泛使用的最大费舍尔信息(MFI)项目选择方法的替代方法,在计算机自适应测试(CAT)中能够有效地平衡题库使用,同时提供准确的潜在特质估计。然而,以往对这些方法的比较将项目参数估计视为真实的总体参数值。因此,可能会出现利用机会的情况。在本文中,我们研究了在更现实的条件下AST方法的性能,即在CAT中使用项目参数而非真实参数值的情况。将其性能与MFI方法在与辛普森 - 赫特或随机曝光控制结合使用时的性能进行了比较。结果表明,MFI方法即使与曝光控制相结合,也容易出现利用机会的情况。在校准样本量较小时尤其如此。另一方面,AST对利用机会的情况更具鲁棒性。与之前使用真实项目参数值的研究一致,AST能使题库使用更加平衡,潜在特质估计精度略有损失。当测试长度达到40个项目时,这种损失可以忽略不计。