Gao Xuliang, Hou Minmin, Wang Fang, Zhou Jinyu
School of Psychology, Guizhou Normal University, Huaxi University Town, Guian New District, Guiyang, 550025, Guizhou Province, China.
School of General Education, Guizhou University of Commerce, Twenty-Sixth Avenue, Maijia Town, Baiyun District, Guiyang, 550001, Guizhou Province, China.
Behav Res Methods. 2025 Apr 9;57(5):138. doi: 10.3758/s13428-025-02659-6.
Assessing person-fit in cognitive diagnostic assessments is a critical research area. Inability to identify misfitting responses can lead to misinterpretation of students' attribute profiles, potentially resulting in incorrect remedial actions. Despite its importance, there is a lack of research on person-fit statistics for polytomous cognitive diagnostic models (CDM). To address this, we propose a new person-fit statistic, WR, specifically designed for polytomous items in CDMs. We evaluated WR's ability to detect three types of abnormal behaviors through simulation studies, comparing its performance with established statistics including l, infit, and outfit. The results show that WR consistently demonstrated stable and superior detection capabilities across all experimental scenarios. Traditional methods showed inconsistent detection abilities for different anomalies; l was more effective at detecting cheating, while infit was better for creative responses. In high-quality test environments, WR performed best, though the difference compared to traditional methods was not significant. However, in low-quality conditions, WR significantly outperformed traditional methods. Overall, WR proved to be an effective tool for detecting person misfit in polytomous scoring CDMs. Finally, we analyzed a real educational assessment data to assess the practical application of WR.
在认知诊断评估中评估个体拟合度是一个关键的研究领域。无法识别不匹配的反应可能会导致对学生属性概况的错误解读，进而可能导致不正确的补救措施。尽管其很重要，但针对多分类认知诊断模型（CDM）的个体拟合度统计研究却很匮乏。为了解决这一问题，我们提出了一种新的个体拟合度统计量WR，它是专门为CDM中的多分类项目设计的。我们通过模拟研究评估了WR检测三种异常行为的能力，并将其性能与包括l、内拟合和外拟合在内的既定统计量进行了比较。结果表明，在所有实验场景中，WR始终表现出稳定且卓越的检测能力。传统方法对不同异常情况的检测能力不一致；l在检测作弊方面更有效，而内拟合在检测创造性反应方面表现更好。在高质量的测试环境中，WR表现最佳，尽管与传统方法相比差异不显著。然而，在低质量条件下，WR明显优于传统方法。总体而言，WR被证明是检测多分类计分CDM中个体不匹配的有效工具。最后，我们分析了真实的教育评估数据以评估WR的实际应用。
