Department of Management, Clemson University.
Department of Management, University of Texas, San Antonio.
J Appl Psychol. 2017 May;102(5):802-828. doi: 10.1037/apl0000193. Epub 2017 Feb 2.
Correcting validity estimates for selection procedures for range restriction typically involves comparing variance in predictor scores between all job applicants and applicants who were selected. However, some research on criterion-related and differential validity of cognitive ability tests has relied on range restriction corrections based on data from job incumbents. Unfortunately, there remains ambiguity concerning the accuracy of this incumbent-based approach vis-à-vis the applicant-based approach. To address this issue, we conducted several Monte Carlo simulations, as well as an analysis of college admissions data. Our first simulation study showed that incumbent-based range restriction corrections result in downwardly biased estimates of criterion-related validity, whereas applicant-based corrections were quite accurate. Our second set of simulations showed that incumbent-based range restriction corrections can produce evidence of differential validity when there is no differential validity in the population. In contrast, applicant-based corrections tended to accurately estimate population parameters and showed little, if any, evidence of differential validity when there is no differential validity in the population. Analysis of data for the ACT as a predictor of academic performance revealed similar patterns of bias for incumbent-based corrections in an academic setting. Overall, the present findings raise serious concerns regarding the use of incumbent-based range restriction corrections in lieu of applicant-based corrections. They also cast doubt on recent evidence for differential validity of predictors of job performance. (PsycINFO Database Record
纠正选择程序中因范围限制导致的有效性估计通常涉及比较所有求职者和被选中的求职者之间预测分数的方差。然而，一些关于认知能力测试的效标关联和差异有效性的研究依赖于基于在职者数据的范围限制校正。不幸的是，关于这种基于在职者的方法相对于基于申请者的方法的准确性仍然存在模糊性。为了解决这个问题，我们进行了几项蒙特卡罗模拟以及对大学招生数据的分析。我们的第一项模拟研究表明，基于在职者的范围限制校正会导致效标关联有效性的估计值向下偏倚，而基于申请者的校正则非常准确。我们的第二项模拟研究表明，基于在职者的范围限制校正可以在人群中没有差异有效性的情况下产生差异有效性的证据。相比之下，基于申请者的校正往往可以准确估计人群参数，并且在人群中没有差异有效性时，几乎没有（如果有的话）差异有效性的证据。对作为学术表现预测指标的 ACT 的数据分析揭示了在学术环境中基于在职者的校正存在类似的偏差模式。总体而言，这些发现对使用基于在职者的范围限制校正代替基于申请者的校正提出了严重的担忧。它们也对最近关于工作绩效预测指标的差异有效性的证据提出了质疑。
