Human Resources Research Organization, Alexandria, Virginia 22314-1591, USA.
J Appl Psychol. 2013 Jan;98(1):114-33. doi: 10.1037/a0030887. Epub 2012 Dec 17.
Though considerable research has evaluated the functioning of assessment center (AC) ratings, surprisingly little research has articulated and uniquely estimated the components of reliable and unreliable variance that underlie such ratings. The current study highlights limitations of existing research for estimating components of reliable and unreliable variance in AC ratings. It provides a comprehensive empirical decomposition of variance in AC ratings that: (a) explicitly accounts for assessee-, dimension-, exercise-, and assessor-related effects, (b) does so with 3 large sets of operational data from a multiyear AC program, and (c) avoids many analytic limitations and confounds that have plagued the AC literature to date. In doing so, results show that (a) the extant AC literature has masked the contribution of sizable, substantively meaningful sources of variance in AC ratings, (b) various forms of assessor bias largely appear trivial, and (c) there is far more systematic, nuanced variance present in AC ratings than previous research indicates. Furthermore, this study also illustrates how the composition of reliable and unreliable variance heavily depends on the level to which assessor ratings are aggregated (e.g., overall AC-level, dimension-level, exercise-level) and the generalizations one desires to make based on those ratings. The implications of this study for future AC research and practice are discussed.
尽管已经有大量研究评估了评估中心（AC）评级的功能，但令人惊讶的是，很少有研究阐明和独特地估计了构成这些评级的可靠和不可靠差异的组成部分。本研究强调了现有研究在估计 AC 评级中可靠和不可靠差异组成部分方面的局限性。它提供了对 AC 评级中差异的全面实证分解，该分解：（a）明确考虑了被评估者、维度、练习和评估者相关的影响，（b）使用来自多年 AC 计划的 3 组大型运营数据来实现这一点，（c）避免了迄今为止困扰 AC 文献的许多分析限制和混淆。这样做的结果表明：（a）现有 AC 文献掩盖了 AC 评级中大量实质性有意义的差异来源的贡献，（b）各种形式的评估者偏见在很大程度上似乎微不足道，（c）AC 评级中存在比以往研究表明的更多系统、细微的差异。此外，这项研究还说明了可靠和不可靠差异的组成如何在很大程度上取决于评估者评级的聚合程度（例如，整体 AC 级别、维度级别、练习级别）以及基于这些评级进行概括的程度。本研究对未来的 AC 研究和实践具有重要意义。
