Crane Paul K, van Belle Gerald, Larson Eric B
Medicine and Public Health and Community Medicine, University of Washington, Seattle 98104, USA.
Stat Med. 2004 Jan 30;23(2):241-56. doi: 10.1002/sim.1713.
Assessment of test bias is important to establish the construct validity of tests. Assessment of differential item functioning (DIF) is an important first step in this process. DIF is present when examinees from different groups have differing probabilities of success on an item, after controlling for overall ability level. Here, we present analysis of DIF in the Cognitive Assessment Screening Instrument (CASI) using data from a large cohort study of elderly adults. We developed an ordinal logistic regression modelling technique to assess test items for DIF. Estimates of cognitive ability were obtained in two ways based on responses to CASI items: using traditional CASI scoring according to the original test instructions as well as using item response theory (IRT) scoring. Several demographic characteristics were examined for potential DIF, including ethnicity and gender (entered into the model as dichotomous variables), and years of education and age (entered as continuous variables). We found that a disappointingly large number of items had DIF with respect to at least one of these demographic variables. More items were found to have DIF with traditional CASI scoring than with IRT scoring. This study demonstrates a powerful technique for the evaluation of DIF in psychometric tests. The finding that so many CASI items had DIF suggests that previous findings of differences between groups in cognitive functioning as measured by the CASI may be due to biased test items rather than true differences between groups. The finding that IRT scoring diminished the impact of DIF is discussed. Some preliminary suggestions for how to deal with items found to have DIF in cognitive tests are made. The advantages of the DIF detection techniques we developed are discussed in relation to other techniques for the evaluation of DIF.
评估测试偏差对于确立测试的结构效度很重要。差异项目功能（DIF）评估是这一过程中的重要第一步。当不同组的考生在控制总体能力水平后在某个项目上成功的概率不同时，就存在DIF。在此，我们使用来自一项针对老年人的大型队列研究的数据，对认知评估筛查工具（CASI）中的DIF进行分析。我们开发了一种有序逻辑回归建模技术来评估测试项目的DIF。基于对CASI项目的回答，通过两种方式获得认知能力估计值：按照原始测试说明使用传统的CASI评分以及使用项目反应理论（IRT）评分。我们考察了几个可能存在DIF的人口统计学特征，包括种族和性别（作为二分变量纳入模型），以及受教育年限和年龄（作为连续变量纳入）。我们发现，数量多得令人失望的项目在至少一个这些人口统计学变量方面存在DIF。与IRT评分相比，发现有更多项目在传统CASI评分时有DIF。本研究展示了一种用于评估心理测量测试中DIF的强大技术。如此多的CASI项目存在DIF这一发现表明，先前通过CASI测量的不同组在认知功能方面的差异发现可能是由于测试项目存在偏差，而非组间的真实差异。文中讨论了IRT评分减少DIF影响这一发现。针对如何处理在认知测试中发现存在DIF的项目提出了一些初步建议。我们开发的DIF检测技术的优势与其他DIF评估技术相关进行了讨论。
