Suppr超能文献

存在校准误差时的分层计算机自适应测试。

-Stratified Computerized Adaptive Testing in the Presence of Calibration Error.

作者信息

Cheng Ying, Patton Jeffrey M, Shao Can

机构信息

University of Notre Dame, Notre Dame, IN, USA.

出版信息

Educ Psychol Meas. 2015 Apr;75(2):260-283. doi: 10.1177/0013164414530719. Epub 2014 Apr 21.

Abstract

-Stratified computerized adaptive testing with -blocking (AST), as an alternative to the widely used maximum Fisher information (MFI) item selection method, can effectively balance item pool usage while providing accurate latent trait estimates in computerized adaptive testing (CAT). However, previous comparisons of these methods have treated item parameter estimates as if they are the true population parameter values. Consequently, capitalization on chance may occur. In this article, we examined the performance of the AST method under more realistic conditions where item parameter instead of true parameter values are used in the CAT. Its performance was compared against that of the MFI method when the latter is used in conjunction with Sympson-Hetter or randomesque exposure control. Results indicate that the MFI method, even when combined with exposure control, is susceptible to capitalization on chance. This is particularly true when the calibration sample size is small. On the other hand, AST is more robust to capitalization on chance. Consistent with previous investigations using true item parameter values, AST yields much more balanced item pool usage, with a small loss in the precision of latent trait estimates. The loss is negligible when the test is as long as 40 items.

摘要

带有分组的分层计算机自适应测试(AST),作为广泛使用的最大费舍尔信息(MFI)项目选择方法的替代方法,在计算机自适应测试(CAT)中能够有效地平衡题库使用,同时提供准确的潜在特质估计。然而,以往对这些方法的比较将项目参数估计视为真实的总体参数值。因此,可能会出现利用机会的情况。在本文中,我们研究了在更现实的条件下AST方法的性能,即在CAT中使用项目参数而非真实参数值的情况。将其性能与MFI方法在与辛普森 - 赫特或随机曝光控制结合使用时的性能进行了比较。结果表明,MFI方法即使与曝光控制相结合,也容易出现利用机会的情况。在校准样本量较小时尤其如此。另一方面,AST对利用机会的情况更具鲁棒性。与之前使用真实项目参数值的研究一致,AST能使题库使用更加平衡,潜在特质估计精度略有损失。当测试长度达到40个项目时,这种损失可以忽略不计。

相似文献

1
-Stratified Computerized Adaptive Testing in the Presence of Calibration Error.
Educ Psychol Meas. 2015 Apr;75(2):260-283. doi: 10.1177/0013164414530719. Epub 2014 Apr 21.
2
Controlling the Minimum Item Exposure Rate in Computerized Adaptive Testing: A Two-Stage Sympson-Hetter Procedure.
Appl Psychol Meas. 2023 Nov;47(7-8):460-477. doi: 10.1177/01466216231209756. Epub 2023 Oct 20.
3
a-Stratified CAT design with content blocking.
Br J Math Stat Psychol. 2003 Nov;56(Pt 2):359-78. doi: 10.1348/000711003770480084.
4
Components of the item selection algorithm in computerized adaptive testing.
J Educ Eval Health Prof. 2018 Mar 24;15:7. doi: 10.3352/jeehp.2018.15.7. eCollection 2018.
5
Computerized adaptive testing: the capitalization on chance problem.
Span J Psychol. 2012 Mar;15(1):424-41. doi: 10.5209/rev_sjop.2012.v15.n1.37348.
6
Comparing single-pool and multiple-pool designs regarding test security in computerized testing.
Behav Res Methods. 2012 Sep;44(3):742-52. doi: 10.3758/s13428-011-0178-5.
7
A simplified version of the maximum information per time unit method in computerized adaptive testing.
Behav Res Methods. 2017 Apr;49(2):502-512. doi: 10.3758/s13428-016-0712-6.
8
A Dynamic Stratification Method for Improving Trait Estimation in Computerized Adaptive Testing Under Item Exposure Control.
Appl Psychol Meas. 2020 May;44(3):182-196. doi: 10.1177/0146621619843820. Epub 2019 Apr 23.
9
Robustness of Adaptive Measurement of Change to Item Parameter Estimation Error.
Educ Psychol Meas. 2022 Aug;82(4):643-677. doi: 10.1177/00131644211033902. Epub 2021 Aug 16.
10
Item selection methods in multidimensional computerized adaptive testing for forced-choice items using Thurstonian IRT model.
Behav Res Methods. 2024 Feb;56(2):600-614. doi: 10.3758/s13428-022-02037-6. Epub 2023 Feb 7.

引用本文的文献

1
Accounting for item calibration error in computerized adaptive testing.
Behav Res Methods. 2025 Mar 26;57(5):126. doi: 10.3758/s13428-025-02649-8.
2
The Impact of Item Calibration Error on Variable-Length Cognitive Diagnostic Computerized Adaptive Testing.
Front Psychol. 2020 Dec 2;11:575141. doi: 10.3389/fpsyg.2020.575141. eCollection 2020.
3
Optimal Online Calibration Designs for Item Replenishment in Adaptive Testing.
Psychometrika. 2020 Mar;85(1):35-55. doi: 10.1007/s11336-019-09687-0. Epub 2019 Sep 17.
4
Ensuring content validity of patient-reported outcomes: a shadow-test approach to their adaptive measurement.
Qual Life Res. 2018 Jul;27(7):1683-1693. doi: 10.1007/s11136-017-1650-1. Epub 2017 Jul 14.

本文引用的文献

1
Stratified Item Selection Methods in Cognitive Diagnosis Computerized Adaptive Testing.
Appl Psychol Meas. 2020 Jul;44(5):346-361. doi: 10.1177/0146621619893783. Epub 2019 Dec 21.
2
Computerized adaptive testing: the capitalization on chance problem.
Span J Psychol. 2012 Mar;15(1):424-41. doi: 10.5209/rev_sjop.2012.v15.n1.37348.
3
Incorporating randomness in the Fisher information for improving item-exposure control in CATs.
Br J Math Stat Psychol. 2008 Nov;61(Pt 2):493-513. doi: 10.1348/000711007X230937. Epub 2007 Aug 4.
4
Controlling item exposure and test overlap on the fly in computerized adaptive testing.
Br J Math Stat Psychol. 2008 Nov;61(Pt 2):471-92. doi: 10.1348/000711007X227067. Epub 2007 Jul 23.
5
a-Stratified CAT design with content blocking.
Br J Math Stat Psychol. 2003 Nov;56(Pt 2):359-78. doi: 10.1348/000711003770480084.

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验