Paulsen Justin, Svetina Dubravka, Feng Yanan, Valdivia Montserrat
Indiana University, Bloomington, USA.
Appl Psychol Meas. 2020 Jun;44(4):267-281. doi: 10.1177/0146621619858675. Epub 2019 Jul 4.
Cognitive diagnostic models (CDMs) are of growing interest in educational research because of the models' ability to provide diagnostic information regarding examinees' strengths and weaknesses suited to a variety of content areas. An important step to ensure appropriate uses and interpretations from CDMs is to understand the impact of differential item functioning (DIF). While methods of detecting DIF in CDMs have been identified, there is a limited understanding of the extent to which DIF affects classification accuracy. This simulation study provides a reference to practitioners to understand how different magnitudes and types of DIF interact with CDM item types and group distributions and sample sizes to influence attribute- and profile-level classification accuracy. The results suggest that attribute-level classification accuracy is robust to DIF of large magnitudes in most conditions, while profile-level classification accuracy is negatively influenced by the inclusion of DIF. Conditions of unequal group distributions and DIF located on simple structure items had the greatest effect in decreasing classification accuracy. The article closes by considering implications of the results and future directions.
认知诊断模型（CDMs）在教育研究中越来越受到关注，因为这些模型能够提供有关考生优势和劣势的诊断信息，适用于各种内容领域。确保对认知诊断模型进行适当使用和解释的重要一步是了解项目功能差异（DIF）的影响。虽然已经确定了在认知诊断模型中检测项目功能差异的方法，但对于项目功能差异在多大程度上影响分类准确性的理解仍然有限。这项模拟研究为从业者提供了一个参考，以了解不同程度和类型的项目功能差异如何与认知诊断模型的项目类型、群体分布和样本大小相互作用，从而影响属性和配置文件级别的分类准确性。结果表明，在大多数情况下，属性级别的分类准确性对大程度的项目功能差异具有鲁棒性，而配置文件级别的分类准确性则受到项目功能差异的负面影响。群体分布不平等和位于简单结构项目上的项目功能差异条件对降低分类准确性的影响最大。文章最后考虑了研究结果的意义和未来方向。
