Zeng Ji, Yin Ping, Shedden Kerby A
Michigan Department of Education, Lansing, MI, USA.
Questar Assessment Inc., Apple Valley, MN, USA.
Educ Psychol Meas. 2015 Dec;75(6):1045-1062. doi: 10.1177/0013164414565006. Epub 2015 Jan 7.
This article provides a brief overview and comparison of three matching approaches in forming comparable groups for a study comparing test administration modes (i.e., computer-based tests [CBT] and paper-and-pencil tests [PPT]): (a) a propensity score matching approach proposed in this article, (b) the propensity score matching approach used by Lottridge, Nicewander, and Mitzel, and (c) a modified approach of matched samples comparability analyses (MSCA) mentioned by Way, Davis, and Fitzpatrick. Different matching approaches resulted in different matched data with differing degrees of matching quality, and matched data from each matching approach were then used in the mode comparison investigation. Construct equivalence was examined and the level of invariance was found to be consistent across modes for all three matching approaches. Raw-to-scale score conversion tables were created, and the impact on CBT students' proficiency classification was examined. The comparison of the number of CBT students whose proficiency classification would be affected and the equality of score distributions between modes on raw scores and scale scores across the three matching approaches indicate that the propensity score matching approach delineated in this article led to the most consistent evidence for the conclusion of the mode comparison.
本文简要概述并比较了三种为比较测试管理模式（即基于计算机的测试[CBT]和纸笔测试[PPT]）而形成可比组的匹配方法：(a)本文提出的倾向得分匹配方法，(b)Lottridge、Nicewander和Mitzel使用的倾向得分匹配方法，以及(c)Way、Davis和Fitzpatrick提到的匹配样本可比性分析（MSCA）的改进方法。不同的匹配方法产生了匹配质量不同的不同匹配数据，然后将每种匹配方法的匹配数据用于模式比较研究。检验了结构等效性，发现所有三种匹配方法在不同模式下的不变性水平是一致的。创建了原始分数到量表分数的转换表，并检验了对CBT学生熟练程度分类的影响。比较三种匹配方法中熟练程度分类会受到影响的CBT学生数量，以及不同模式在原始分数和量表分数上的分数分布平等性，结果表明本文所描述的倾向得分匹配方法为模式比较结论提供了最一致的证据。
