School of Information Science and Technology, Northeast Normal University, 2555 Jingyue Jilin, 130117, Changchun, China.
Center for Studies of Education and Psychology of Ethnic Minorities in Southwest China, Southwest University, 2 Tiansheng, Beibei, 400715, Chongqing, China.
Behav Res Methods. 2024 Oct;56(7):6981-7004. doi: 10.3758/s13428-024-02404-5. Epub 2024 Apr 30.
Cognitive diagnosis is a crucial element of intelligent education that aims to assess the proficiency of specific skills or traits in students at a refined level and provide insights into their strengths and weaknesses for personalized learning. Researchers have developed numerous cognitive diagnostic models. However, previous studies indicate that diagnostic accuracy can be significantly influenced by the appropriateness of the model and the sample size. Thus, designing a general model that can adapt to different assumptions and sample sizes remains a considerable challenge. Artificial neural networks have been proposed as a promising approach in some studies. In this paper, we propose a cognitive diagnosis model of a neural network constrained by a Q-matrix and named QNN. Specifically, we employ the Q-matrix to determine the connections between neurons and the width and depth of the neural network. Moreover, to reduce the human effort in the training algorithm, we designed a self-organizing map-based cognitive diagnosis training framework called SOM-NN, which enables the QNN to be trained unsupervised. Extensive experimental results on simulated and real datasets demonstrate that our approaches are effective in both accuracy and interpretability. Notably, under unsupervised conditions, our approach has significant advantages on small sample datasets with high levels of guessing and slipping, especially on the pattern-wise agreement rates. This work bridges the gap between psychometrics and machine learning and provides a realistic and implementable reference solution for classroom instructional assessment and the cold start of personalized and adaptive assessment systems.
认知诊断是智能教育的一个重要组成部分,旨在以更精细的水平评估学生特定技能或特质的熟练程度,并深入了解他们的优势和劣势,以实现个性化学习。研究人员已经开发出许多认知诊断模型。然而,先前的研究表明,诊断准确性会受到模型和样本大小的适当性的显著影响。因此,设计一个能够适应不同假设和样本大小的通用模型仍然是一个相当大的挑战。在一些研究中,人工神经网络被提出作为一种有前途的方法。在本文中,我们提出了一种受 Q 矩阵约束的神经网络认知诊断模型,称为 QNN。具体来说,我们使用 Q 矩阵来确定神经元之间的连接以及神经网络的宽度和深度。此外,为了减少训练算法中的人为努力,我们设计了一个基于自组织映射的认知诊断训练框架,称为 SOM-NN,它可以使 QNN 进行无监督训练。在模拟和真实数据集上的广泛实验结果表明,我们的方法在准确性和可解释性方面都很有效。值得注意的是,在无监督条件下,我们的方法在具有高水平猜测和滑动的小样本数据集上具有显著优势,尤其是在模式一致性率方面。这项工作弥合了心理测量学和机器学习之间的差距,为课堂教学评估和个性化及自适应评估系统的冷启动提供了一个现实可行的参考解决方案。