a Educational Testing Service.
b University of California , Berkeley.
Multivariate Behav Res. 2019 May-Jun;54(3):360-381. doi: 10.1080/00273171.2018.1530091. Epub 2019 Mar 28.
In this study we extend and assess the trifactor model for multiple-ratings data in which two different raters give independent scores for the same responses (e.g., in the GRE essay or to subset of PISA constructed-responses). The trifactor model was extended to incorporate a cross-classified data structure (e.g., items and raters) instead of a strictly hierarchical structure. we present a set of simulations to reflect the incompleteness and imbalance in real-world assessments. The effects of the rate of missingness in the data and of ignoring differences among raters are investigated using two sets of simulations. The use of the trifactor model is also illustrated with empirical data analysis using a well-known international large-scale assessment.
在这项研究中，我们扩展并评估了三因素模型，用于多评分者数据，其中两个不同的评分者对相同的反应（例如，在 GRE 作文或 PISA 建构反应的子集中）给出独立的分数。三因素模型扩展为包含交叉分类数据结构（例如，项目和评分者），而不是严格的层次结构。我们呈现了一系列模拟，以反映现实评估中的不完整性和不平衡性。使用两组模拟研究了数据中缺失率和忽略评分者之间差异的影响。还使用一个著名的国际大规模评估的实证数据分析来说明三因素模型的使用。
