Crişan Daniela R, Tendeiro Jorge N, Meijer Rob R
University of Groningen, The Netherlands.
Appl Psychol Meas. 2017 Sep;41(6):439-455. doi: 10.1177/0146621617695522. Epub 2017 Mar 17.
In this article, the consequences of violations of unidimensionality on selection decisions in the framework of unidimensional item response theory (IRT) models are investigated based on simulated data. The factors manipulated include the severity of violations, the proportion of misfitting items, and test length. The outcomes that were considered are the precision and accuracy of the estimated model parameters, the correlations of estimated ability ([Formula: see text]) and number-correct ([Formula: see text]) scores with the true ability ([Formula: see text]), the ranks of the examinees and the overlap between sets of examinees selected based on either [Formula: see text], [Formula: see text], or [Formula: see text] scores, and the bias in criterion-related validity estimates. Results show that the [Formula: see text] values were unbiased by violations of unidimensionality, but their precision decreased as multidimensionality and the proportion of misfitting items increased; the estimated item parameters were robust to violations of unidimensionality. The correlations between [Formula: see text], [Formula: see text], and [Formula: see text] scores, the agreement between the three selection criteria, and the accuracy of criterion-related validity estimates are all negatively affected, to some extent, by increasing levels of multidimensionality and the proportion of misfitting items. However, removing the misfitting items only improved the results in the case of severe multidimensionality and large proportion of misfitting items, and deteriorated them otherwise.
在本文中，基于模拟数据研究了在单维项目反应理论（IRT）模型框架下违反单维性对选择决策的影响。所操纵的因素包括违反的严重程度、不拟合项目的比例和测验长度。所考虑的结果包括估计模型参数的精度和准确性、估计能力（[公式：见原文]）和答对题数（[公式：见原文]）分数与真实能力（[公式：见原文]）的相关性、考生的排名以及基于[公式：见原文]、[公式：见原文]或[公式：见原文]分数选择的考生集合之间的重叠情况，以及与标准相关的效度估计中的偏差。结果表明，[公式：见原文]值不受违反单维性的影响，但随着多维性和不拟合项目比例的增加，其精度会降低；估计的项目参数对违反单维性具有稳健性。[公式：见原文]、[公式：见原文]和[公式：见原文]分数之间的相关性、三种选择标准之间的一致性以及与标准相关的效度估计的准确性在一定程度上都受到多维性水平和不拟合项目比例增加的负面影响。然而，去除不拟合项目仅在严重多维性和不拟合项目比例较大的情况下改善了结果，否则会使其恶化。
