Rios Joseph A, Deng Jiayi
Department of Educational Psychology, University of Minnesota, Minneapolis, MN.
Appl Psychol Meas. 2022 Jan;46(1):40-52. doi: 10.1177/01466216211051719. Epub 2021 Oct 11.
An underlying threat to the validity of reliability measures is the introduction of systematic variance in examinee scores from unintended constructs that differ from those assessed. One construct-irrelevant behavior that has gained increased attention in the literature is rapid guessing (RG), which occurs when examinees answer quickly with intentional disregard for item content. To examine the degree of distortion in coefficient alpha due to RG, this study compared alpha estimates between conditions in which simulees engaged in full solution (i.e., do not engage in RG) versus partial RG behavior. This was done by conducting a simulation study in which the percentage and ability characteristics of rapid responders as well as the percentage and pattern of RG were manipulated. After controlling for test length and difficulty, the average degree of distortion in estimates of coefficient alpha due to RG ranged from -.04 to .02 across 144 conditions. Although slight differences were noted between conditions differing in RG pattern and RG responder ability, the findings from this study suggest that estimates of coefficient alpha are largely robust to the presence of RG due to cognitive fatigue and a low perceived probability of success.
可靠性测量有效性的一个潜在威胁是，考生分数中引入了来自与所评估内容不同的意外结构的系统方差。文献中越来越受关注的一种与结构无关的行为是快速猜测（RG），即考生故意不顾题目内容而快速作答。为了研究快速猜测导致的系数α的扭曲程度，本研究比较了模拟考生进行完整解答（即不进行快速猜测）与部分快速猜测行为这两种情况下的α估计值。这是通过进行一项模拟研究来实现的，在该研究中，对快速答题者的百分比和能力特征以及快速猜测的百分比和模式进行了操纵。在控制了测验长度和难度后，在144种情况下，快速猜测导致的系数α估计值的平均扭曲程度在-.04至.02之间。尽管在快速猜测模式和快速猜测者能力不同的情况之间发现了细微差异，但本研究的结果表明，由于认知疲劳和成功的感知概率较低，系数α的估计值在很大程度上不受快速猜测的影响。
