Chen Cheng-Te, Hwu Bo-Sien
National Tsing Hua University, Hsinchu, Taiwan.
National Sun Yat-sen University, Kaohsiung, Taiwan.
Appl Psychol Meas. 2018 May;42(3):206-220. doi: 10.1177/0146621617726786. Epub 2017 Aug 29.
By design, large-scale educational testing programs often have a large proportion of missing data. Since the effect of missing data on differential item functioning (DIF) assessment has been investigated in recent years and it has been found that Type I error rates tend to be inflated, it is of great importance to adapt existing DIF assessment methods to the inflation. The DIF-free-then-DIF (DFTD) strategy, which originally involved one single-scale purification procedure to identify DIF-free items, has been extended to involve another scale purification procedure for the DIF assessment in this study, and this new method is called the dual-scale purification (DSP) procedure. The performance of the DSP procedure in assessing DIF in large-scale programs, such as Program for International Student Assessment (PISA), was compared with the DFTD strategy through a series of simulation studies. Results showed the superiority of the DSP procedure over the DFTD strategy when tests consisted of many DIF items and when data were missing by design as in large-scale programs. Moreover, an empirical study of the PISA 2009 Taiwan sample was provided to show the implications of the DSP procedure. The applications as well as further studies of DSP procedure are also discussed.
从设计角度来看，大规模教育测试项目往往存在大量缺失数据。近年来，由于缺失数据对项目功能差异（DIF）评估的影响已得到研究，并且发现第一类错误率往往会被夸大，因此使现有的DIF评估方法适应这种夸大情况至关重要。“无DIF然后DIF”（DFTD）策略最初涉及一个单一量表净化程序以识别无DIF项目，在本研究中已扩展为涉及另一个用于DIF评估的量表净化程序，这种新方法称为双量表净化（DSP）程序。通过一系列模拟研究，将DSP程序在评估大规模项目（如国际学生评估项目（PISA））中的DIF时的表现与DFTD策略进行了比较。结果表明，当测试包含许多DIF项目且数据如大规模项目那样因设计而缺失时，DSP程序优于DFTD策略。此外，还提供了对2009年PISA台湾样本的实证研究，以展示DSP程序的意义。同时也讨论了DSP程序的应用以及进一步的研究。
