Department of Biochemistry and Molecular Cell Biology, University Medical Center Hamburg-Eppendorf, Martinistraße 52, D-20246, Hamburg, Germany.
BMC Med Educ. 2017 Dec 11;17(1):246. doi: 10.1186/s12909-017-1070-5.
The validity of selection tests is underestimated if it is determined by simply calculating the predictor-outcome correlation found in the admitted group. This correlation is usually attenuated by two factors: (1) the combination of selection variables which can compensate for each other and (2) range restriction in predictor and outcome due to the absence of outcome measures for rejected applicants.
Here we demonstrate the logic of these artifacts in a situation typical for student selection tests and compare four different methods for their correction: two formulas for the correction of direct and indirect range restriction, expectation maximization algorithm (EM) and multiple imputation by chained equations (MICE). First we show with simulated data how a realistic estimation of predictive validity could be achieved; second we apply the same methods to empirical data from one medical school.
The results of the four methods are very similar except for the direct range restriction formula which underestimated validity.
For practical purposes Thorndike's case C formula is a relatively straightforward solution to the range restriction problem, provided distributional assumptions are met. With EM and MICE more precision is obtained when distributional requirements are not met, but access to a sophisticated statistical package such as R is needed. The use of true score correlation has its own problems and does not seem to provide a better correction than other methods.
如果仅通过计算录取组中发现的预测变量与结果的相关系数来确定选拔测试的有效性，则会低估其有效性。该相关性通常会受到两个因素的减弱：（1）选择变量的组合可以相互补偿；（2）由于拒绝申请人没有结果测量，预测因子和结果受到范围限制。
在这里，我们在典型的学生选拔测试情况下展示了这些伪影的逻辑，并比较了四种不同的校正方法：两种用于直接和间接范围限制的校正公式、期望最大化算法（EM）和通过链式方程的多重插补（MICE）。首先，我们使用模拟数据展示了如何实现现实预测有效性的估计；其次，我们将相同的方法应用于一所医学院的实证数据。
除了直接范围限制公式低估了有效性之外，这四种方法的结果非常相似。
对于实际目的，只要满足分布假设，桑代克的案例 C 公式就是解决范围限制问题的相对直接的解决方案。在不满足分布要求的情况下，使用 EM 和 MICE 可以获得更高的精度，但需要访问 R 等复杂的统计软件包。真分数相关的使用有其自身的问题，并且似乎并不比其他方法提供更好的校正。
