Wang Jue, Engelhard George, Wolfe Edward W
The University of Georgia, Athens, GA, USA.
Pearson, Iowa City, IA, USA.
Educ Psychol Meas. 2016 Dec;76(6):1005-1025. doi: 10.1177/0013164415621606. Epub 2015 Dec 14.
The number of performance assessments continues to increase around the world, and it is important to explore new methods for evaluating the quality of ratings obtained from raters. This study describes an unfolding model for examining rater accuracy. Accuracy is defined as the difference between observed and expert ratings. Dichotomous accuracy ratings (0 = inaccurate, 1 = accurate) are unfolded into three latent categories: inaccurate below expert ratings, accurate ratings, and inaccurate above expert ratings. The hyperbolic cosine model (HCM) is used to examine dichotomous accuracy ratings from a statewide writing assessment. This study suggests that HCM is a promising approach for examining rater accuracy, and that the HCM can provide a useful interpretive framework for evaluating the quality of ratings obtained within the context of rater-mediated assessments.
全球范围内绩效评估的数量持续增加，探索评估评分者所给评分质量的新方法很重要。本研究描述了一种用于检验评分者准确性的展开模型。准确性被定义为观察到的评分与专家评分之间的差异。二分制准确性评分（0 = 不准确，1 = 准确）被展开为三个潜在类别：低于专家评分的不准确、准确评分以及高于专家评分的不准确。双曲余弦模型（HCM）用于检验来自全州写作评估的二分制准确性评分。本研究表明，双曲余弦模型是检验评分者准确性的一种有前景的方法，并且双曲余弦模型可以为评估在评分者介导的评估背景下所获得评分的质量提供一个有用的解释框架。
