Koopman Letty, Braeken Johan
University of Groningen, The Netherlands.
University of Oslo, Norway.
Educ Psychol Meas. 2024 Sep 6:00131644241274122. doi: 10.1177/00131644241274122.
Educational and psychological tests with an ordered item structure enable efficient test administration procedures and allow for intuitive score interpretation and monitoring. The effectiveness of the measurement instrument relies to a large extent on the validated strength of its ordering structure. We define three increasingly strict types of ordering for the ordering structure of a measurement instrument with clustered items: a weak and a strong invariant cluster ordering and a clustered invariant item ordering. Following a nonparametric item response theory (IRT) approach, we proposed a procedure to evaluate the ordering structure of a clustered item set along this three-fold continuum of order invariance. The basis of the procedure is (a) the local assessment of pairwise conditional expectations at both cluster and item level and (b) the global assessment of the number of Guttman errors through new generalizations of the -coefficient for this item-cluster context. The procedure, readily implemented in R, is illustrated and applied to an empirical example. Suggestions for test practice, further methodological developments, and future research are discussed.
具有有序项目结构的教育和心理测试能够实现高效的测试管理程序，并允许进行直观的分数解释和监测。测量工具的有效性在很大程度上依赖于其有序结构的有效强度。我们为具有聚类项目的测量工具的有序结构定义了三种越来越严格的排序类型：弱不变聚类排序和强不变聚类排序以及聚类不变项目排序。遵循非参数项目反应理论（IRT）方法，我们提出了一种程序，以沿着这种三重顺序不变性连续体评估聚类项目集的有序结构。该程序的基础是：（a）在聚类和项目层面进行成对条件期望的局部评估，以及（b）通过针对此项目聚类背景对 - 系数的新推广来全局评估古特曼误差的数量。该程序很容易在R中实现，并通过一个实证例子进行说明和应用。还讨论了对测试实践的建议、进一步的方法学发展以及未来的研究。
