Quirk Victoria L, Kern Justin L
Department of Educational Psychology, University of Illinois at Urbana-Champaign, Champaign, IL 61820, USA.
J Intell. 2023 Nov 17;11(11):216. doi: 10.3390/jintelligence11110216.
The measurement of psychological constructs is frequently based on self-report tests, which often have Likert-type items rated from "Strongly Disagree" to "Strongly Agree". Recently, a family of item response theory (IRT) models called IRTree models have emerged that can parse out content traits (e.g., personality traits) from noise traits (e.g., response styles). In this study, we compare the selection validity and adverse impact consequences of noise traits on selection when scores are estimated using a generalized partial credit model (GPCM) or an IRTree model. First, we present a simulation which demonstrates that when noise traits do exist, the selection decisions made based on the IRTree model estimated scores have higher accuracy rates and have less instances of adverse impact based on extreme response style group membership when compared to the GPCM. Both models performed similarly when there was no influence of noise traits on the responses. Second, we present an application using data collected from the Open-Source Psychometrics Project Fisher Temperament Inventory dataset. We found that the IRTree model had a better fit, but a high agreement rate between the model decisions resulted in virtually identical impact ratios between the models. We offer considerations for applications of the IRTree model and future directions for research.
心理构念的测量通常基于自我报告测试，这类测试往往有从“强烈不同意”到“强烈同意”进行评分的李克特式项目。最近，出现了一类称为IRTree模型的项目反应理论（IRT）模型，它可以从噪声特质（如反应风格）中解析出内容特质（如人格特质）。在本研究中，我们比较了使用广义部分计分模型（GPCM）或IRTree模型估计分数时，噪声特质对选拔的选择效度和不利影响后果。首先，我们进行了一项模拟，结果表明当存在噪声特质时，与GPCM相比，基于IRTree模型估计分数做出的选拔决策具有更高的准确率，并且基于极端反应风格组成员身份的不利影响情况更少。当噪声特质对反应没有影响时，两个模型的表现相似。其次，我们展示了一个使用从开源心理测量项目费舍尔气质量表数据集中收集的数据的应用。我们发现IRTree模型拟合得更好，但模型决策之间的高一致率导致模型之间的影响比率几乎相同。我们为IRTree模型的应用提供了考虑因素以及未来的研究方向。
