Pacific Metrics, 1 Lower Ragsdale Dr #150, Monterey, CA, 93940, USA.
Psychometrika. 2017 Jun;82(2):498-522. doi: 10.1007/s11336-017-9553-1. Epub 2017 Mar 13.
Parameter recovery and item utilization were investigated for different designs for online test item calibration. The design was adaptive in a double sense: it assumed both adaptive testing of examinees from an operational pool of previously calibrated items and adaptive assignment of field-test items to the examinees. Four criteria of optimality for the assignment of the field-test items were used, each of them based on the information in the posterior distributions of the examinee's ability parameter during adaptive testing as well as the sequentially updated posterior distributions of the field-test item parameters. In addition, different stopping rules based on target values for the posterior standard deviations of the field-test parameters and the size of the calibration sample were used. The impact of each of the criteria and stopping rules on the statistical efficiency of the estimates of the field-test parameters and on the time spent by the items in the calibration procedure was investigated. Recommendations as to the practical use of the designs are given.
针对在线测试项目校准的不同设计，研究了参数恢复和项目利用情况。该设计具有双重适应性：它假设了对来自先前校准项目的操作池中的考生进行自适应测试，以及对考生进行现场测试项目的自适应分配。使用了四项最优分配现场测试项目的标准，每项标准都基于自适应测试过程中考生能力参数的后验分布以及现场测试项目参数的顺序更新后验分布中的信息。此外，还使用了基于现场测试参数后验标准差和校准样本大小的目标值的不同停止规则。研究了每个标准和停止规则对现场测试参数估计的统计效率以及项目在校准过程中花费的时间的影响。给出了有关设计实际使用的建议。
