Nanjing University of Information Science and Technology, China.
Beijing Normal University, China.
Br J Math Stat Psychol. 2021 Jul;74 Suppl 1:24-51. doi: 10.1111/bmsp.12221. Epub 2020 Oct 12.
Previous designs for online calibration have only considered examinees' responses to items. However, the use of response time, a useful metric that can easily be collected by a computer, has not yet been embedded in calibration designs. In this article we utilize response time to optimize the assignment of new items online, and accordingly propose two new adaptive designs. These are the D-optimal per expectation time unit design (D-ET) and the D-optimal per time unit design (D-T). The former method uses the conditional maximum likelihood estimation (CMLE) method to estimate the expected response times, while the latter employs the nonparametric k-nearest-neighbour method to predict the response times. Simulations were conducted to compare the two new designs with the D-optimal online calibration design (D design) in the context of continuous online calibration. In addition, a preliminary study was carried out to evaluate the performance of CMLE prior to its application in D-ET. The results showed that, compared to the D design, the D-ET and D-T designs saved response time and accrued larger calibration information per time unit, without sacrificing item calibration precision.
先前的在线标定设计仅考虑了应试者对项目的反应。然而，反应时间的使用——一个可以通过计算机轻松收集的有用指标——尚未嵌入到标定设计中。在本文中，我们利用反应时间来优化在线分配新的项目，并相应地提出了两种新的自适应设计。这两种设计分别是基于期望时间单位的 D-最优设计（D-ET）和基于时间单位的 D-最优设计（D-T）。前者使用条件最大似然估计（CMLE）方法来估计预期的反应时间，而后者则使用非参数 k-最近邻方法来预测反应时间。通过连续在线标定的情况，进行了模拟比较这两种新设计与 D-最优在线标定设计（D 设计）的性能。此外，进行了初步研究，以评估在应用于 D-ET 之前，CMLE 的性能。结果表明，与 D 设计相比，D-ET 和 D-T 设计节省了反应时间，并且在不牺牲项目标定精度的情况下，每个时间单位累积了更多的标定信息。
