Educational Testing Service, Princeton, NJ, USA.
Behav Res Methods. 2022 Feb;54(1):117-132. doi: 10.3758/s13428-021-01615-4. Epub 2021 Jun 9.
We propose a Sankey diagram-based visualization method to illustrate test-takers' action sequences in technology-enhanced, drag-and-drop items. This method extends existing studies focusing only on initial or last attempt(s) of answer formulation, and extracts, at the item level, comprehensive information of test-takers' drag-and-drop actions. Using the process data from three mathematical items in the National Assessment of Educational Progress, we exemplify how to develop Sankey diagrams, discover frequent actions at each stage of problem solving, deduce test-takers' response strategies, and discuss the relationship among domain (i.e., mathematics) knowledge, response strategy, and item performance. The applications and future extensions of this method can contribute to process-based problem-solving research in educational assessments.
我们提出了一种基于桑基图的可视化方法,用于说明在增强技术的拖放项目中测试者的操作序列。这种方法扩展了仅关注答案形成的初始或最后尝试的现有研究,并从项目级别提取了测试者拖放操作的全面信息。使用全国教育进展评估中三个数学项目的过程数据,我们举例说明了如何开发桑基图,发现解决问题每个阶段的常见操作,推断测试者的反应策略,并讨论领域(即数学)知识、反应策略和项目表现之间的关系。这种方法的应用和未来扩展可以为教育评估中的基于过程的解决问题研究做出贡献。
Behav Res Methods. 2022-2
Br J Educ Psychol. 2017-6
J Exp Child Psychol. 2000-3
J Appl Meas. 2017
J Exp Psychol Appl. 2015-6
IEEE Comput Graph Appl. 2021