Northeast Normal University, Changchun, China.
University of Connecticut, Storrs, , CT, 06250, USA.
Psychometrika. 2022 Dec;87(4):1290-1317. doi: 10.1007/s11336-022-09845-x. Epub 2022 Mar 29.
Computerized assessment provides rich multidimensional data including trial-by-trial accuracy and response time (RT) measures. A key question in modeling this type of data is how to incorporate RT data, for example, in aid of ability estimation in item response theory (IRT) models. To address this, we propose a joint model consisting of a two-parameter IRT model for the dichotomous item response data, a log-normal model for the continuous RT data, and a normal model for corresponding paper-and-pencil scores. Then, we reformulate and reparameterize the model to capture the relationship between the model parameters, to facilitate the prior specification, and to make the Bayesian computation more efficient. Further, we propose several new model assessment criteria based on the decomposition of deviance information criterion (DIC) the logarithm of the pseudo-marginal likelihood (LPML). The proposed criteria can quantify the improvement in the fit of one part of the multidimensional data given the other parts. Finally, we have conducted several simulation studies to examine the empirical performance of the proposed model assessment criteria and have illustrated the application of these criteria using a real dataset from a computerized educational assessment program.
计算机化评估提供了丰富的多维数据，包括逐次试验的准确性和反应时间（RT）测量。在对这类数据进行建模时的一个关键问题是如何结合 RT 数据，例如，在项目反应理论（IRT）模型中帮助进行能力估计。为了解决这个问题，我们提出了一个联合模型，由二参数 IRT 模型用于二项式项目反应数据，对数正态模型用于连续 RT 数据，以及正态模型用于相应的纸笔分数。然后，我们重新制定和重新参数化模型，以捕捉模型参数之间的关系，便于先验指定，并使贝叶斯计算更加高效。此外，我们还提出了几种基于偏差信息准则（DIC）分解的新模型评估标准，以及伪边际似然对数（LPML）。所提出的标准可以量化在给定其他部分的情况下，多维数据的一部分拟合度的提高。最后，我们进行了几项模拟研究，以检验所提出的模型评估标准的实证性能，并使用来自计算机化教育评估程序的真实数据集说明了这些标准的应用。
