Department of Statistics, Stockholm University, 10691, Stockholm, Sweden.
Department of Computer and Information Science, Linköping University, 58183, Linköping, Sweden.
Psychometrika. 2024 Sep;89(3):903-928. doi: 10.1007/s11336-024-09968-3. Epub 2024 Apr 15.
When large achievement tests are conducted regularly, items need to be calibrated before being used as operational items in a test. Methods have been developed to optimally assign pretest items to examinees based on their abilities. Most of these methods, however, are intended for situations where examinees arrive sequentially to be assigned to calibration items. In several calibration tests, examinees take the test simultaneously or in parallel. In this article, we develop an optimal calibration design tailored for such parallel test setups. Our objective is both to investigate the efficiency gain of the method as well as to demonstrate that this method can be implemented in real calibration scenarios. For the latter, we have employed this method to calibrate items for the Swedish national tests in Mathematics. In this case study, like in many real test situations, items are of mixed format and the optimal design method needs to handle that. The method we propose works for mixed-format tests and accounts for varying expected response times. Our investigations show that the proposed method considerably enhances calibration efficiency.
当定期进行大型成就测试时，需要在将项目用作测试中的操作项目之前对其进行校准。已经开发了各种方法，以便根据考生的能力将预测试项目最佳地分配给考生。然而，这些方法中的大多数都是针对考生按顺序到达以分配给校准项目的情况。在几次校准测试中，考生同时或并行参加考试。在本文中，我们为这种并行测试设置开发了一种定制的最佳校准设计。我们的目标既是为了研究该方法的效率增益，也是为了证明该方法可以在实际的校准场景中实施。对于后者，我们已经将这种方法用于校准瑞典国家数学考试的项目。在这个案例研究中，就像在许多实际的测试情况中一样，项目具有混合格式，并且最佳设计方法需要处理这种情况。我们提出的方法适用于混合格式的测试，并考虑了不同的预期响应时间。我们的研究表明，所提出的方法大大提高了校准效率。
