Department of Statistics, London School of Economics and Political Science, London, UK.
Psychometrika. 2020 Dec;85(4):1052-1075. doi: 10.1007/s11336-020-09734-1. Epub 2020 Dec 21.
Problem solving has been recognized as a central skill that today's students need to thrive and shape their world. As a result, the measurement of problem-solving competency has received much attention in education in recent years. A popular tool for the measurement of problem solving is simulated interactive tasks, which require students to uncover some of the information needed to solve the problem through interactions with a computer-simulated environment. A computer log file records a student's problem-solving process in details, including his/her actions and the time stamps of these actions. It thus provides rich information for the measurement of students' problem-solving competency. On the other hand, extracting useful information from log files is a challenging task, due to its complex data structure. In this paper, we show how log file process data can be viewed as a marked point process, based on which we propose a continuous-time dynamic choice model. The proposed model can serve as a measurement model for scaling students along the latent traits of problem-solving competency and action speed, based on data from one or multiple tasks. A real data example is given based on data from Program for International Student Assessment 2012.
问题解决能力已被公认为是当今学生需要具备的核心技能，以茁壮成长并塑造他们的世界。因此，近年来教育界非常关注问题解决能力的衡量。一种流行的问题解决能力衡量工具是模拟互动任务，它要求学生通过与计算机模拟环境进行交互来发现解决问题所需的一些信息。学生的问题解决过程会被计算机日志文件详细记录下来，包括他/她的操作以及这些操作的时间戳。因此，日志文件为衡量学生的问题解决能力提供了丰富的信息。另一方面，由于其复杂的数据结构，从日志文件中提取有用信息是一项具有挑战性的任务。在本文中，我们展示了如何将日志文件处理数据视为标记点过程，在此基础上提出了一个连续时间动态选择模型。该模型可用于根据一项或多项任务的数据，沿着问题解决能力和动作速度的潜在特征对学生进行定标。基于 2012 年国际学生评估计划的数据，给出了一个真实数据示例。
