Department of Psychology, University of Notre Dame, Notre Dame, Indiana, USA.
Jiangxi Normal University, Nanchang, Jiangxi, China.
Br J Math Stat Psychol. 2020 Nov;73 Suppl 1:145-179. doi: 10.1111/bmsp.12191. Epub 2019 Nov 25.
In a cognitive diagnostic assessment (CDA), attributes refer to fine-grained knowledge points or skills. The Q-matrix is a central component of CDA, which specifies the relationship between items and attributes. Oftentimes, attributes and Q-matrix are defined by subject-matter experts, and assumed to be appropriate without any misspecifications. However, this assumption does not always hold in real applications. To address this concern, this paper proposes a residual-based statistic for validating the Q-matrix. Its performance is evaluated in a simulation study and compared against that of an existing method proposed in Liu, Xu and Ying (2012, Applied Psychological Measurement, 36, 548). Simulation results indicate that the proposed method leads to a higher recovery rate of the Q-matrix and is computationally more efficient. The advantage in computational efficiency is particularly pronounced when the number of attributes measured by the test reaches five or more. Results also suggest that the two methods have different tendencies in estimating the attribute vector for each item. In cases where the methods fail to recover the correct Q-matrix, the method in Liu et al. (2012, Applied Psychological Measurement, 36, 548) tends to overestimate the number of attributes measured by the items, whereas our method does not show that bias.
在认知诊断评估（CDA）中，属性是指细粒度的知识点或技能。Q 矩阵是 CDA 的核心组成部分，它指定了项目和属性之间的关系。通常，属性和 Q 矩阵由主题专家定义，并假设没有任何错误指定。然而，这种假设并不总是在实际应用中成立。为了解决这个问题，本文提出了一种基于残差的统计量来验证 Q 矩阵。在模拟研究中评估了其性能，并与 Liu、Xu 和 Ying（2012，应用心理测量，36，548）提出的现有方法进行了比较。模拟结果表明，所提出的方法导致 Q 矩阵的恢复率更高，并且计算效率更高。当测试测量的属性数量达到五个或更多时，计算效率的优势尤其明显。结果还表明，这两种方法在估计每个项目的属性向量方面具有不同的趋势。在方法无法恢复正确的 Q 矩阵的情况下，Liu 等人（2012，应用心理测量，36，548）提出的方法往往会高估项目测量的属性数量，而我们的方法则没有表现出这种偏差。
