Lugu Benjamin, Guo Wenjing, Ma Wenchao
Department of Educational Studies in Psychology, Research Methodology, and Counseling, The University of Alabama, Tuscaloosa, AL, 35401, USA.
Pearson Inc., Suite 301, 927 E. Sonterra Blvd, San Antonio, TX, 78258, USA.
Behav Res Methods. 2025 Jul 1;57(8):213. doi: 10.3758/s13428-025-02734-y.
Cognitive diagnosis assessments are frequently used for formative purposes. Due to the low-stakes nature of these assessments, students may exhibit disengaged behaviors, such as rapid guessing and item omissions. Most existing studies in cognitive diagnosis models assume that item responses are reflections of students' proficiency without considering their engagement levels. This study proposes a disengaged behavior cognitive diagnosis model (DB-CDM) that accounts for both disengaged and engaged behaviors simultaneously. We examined the performance of the DB-CDM through simulation and empirical studies. The simulation showed that the item parameters of the DB-CDM were recovered well, especially when the sample size was large and the proportion of disengaged students was small. The DB-CDM can also accurately identify disengaged students, even under some unfavorable conditions involving a large number of disengaged students. By comparing DB-CDM with the compensatory reparameterized unified model in terms of attribute classifications, we observed that the DB-CDM yielded similar if not higher attribute classifications. In the real data analysis, we found that engaged students had a lower probability of omission and guessing and a higher probability of exhibiting solution behavior compared to disengaged students. This paper provides some initial evidence to support the use of DB-CDM when disengaged behaviors occur.
认知诊断评估经常用于形成性目的。由于这些评估的低风险性质，学生可能会表现出不参与行为，如快速猜测和题目遗漏。认知诊断模型中的大多数现有研究假设题目回答是学生能力水平的反映，而没有考虑他们的参与程度。本研究提出了一种同时考虑不参与和参与行为的不参与行为认知诊断模型（DB-CDM）。我们通过模拟和实证研究检验了DB-CDM的性能。模拟结果表明，DB-CDM的题目参数恢复良好，特别是当样本量较大且不参与学生的比例较小时。即使在有大量不参与学生的一些不利条件下，DB-CDM也能准确识别不参与的学生。通过在属性分类方面将DB-CDM与补偿性重新参数化统一模型进行比较，我们观察到DB-CDM产生了相似甚至更高的属性分类。在实际数据分析中，我们发现与不参与的学生相比，参与的学生出现遗漏和猜测的概率较低，表现出解决问题行为的概率较高。本文提供了一些初步证据，支持在出现不参与行为时使用DB-CDM。
