Paek Insu
Florida State University, Tallahassee, USA.
Appl Psychol Meas. 2015 Jun;39(4):264-277. doi: 10.1177/0146621614559516. Epub 2014 Dec 16.
Guessing is known to influence the test reliability of multiple-choice tests. Although there are many studies that have examined the impact of guessing, they used rather restrictive assumptions (e.g., parallel test assumptions, homogeneous inter-item correlations, homogeneous item difficulty, and homogeneous guessing levels across items) to evaluate the relation between guessing and test reliability. Based on the item response theory (IRT) framework, this study investigated the extent of the impact of guessing on reliability under more realistic conditions where item difficulty, item discrimination, and guessing levels actually vary across items with three different test lengths (TL). By accommodating multiple item characteristics simultaneously, this study also focused on examining interaction effects between guessing and other variables entered in the simulation to be more realistic. The simulation of the more realistic conditions and calculations of reliability and classical test theory (CTT) item statistics were facilitated by expressing CTT item statistics, coefficient α, and reliability in terms of IRT model parameters. In addition to the general negative impact of guessing on reliability, results showed interaction effects between TL and guessing and between guessing and test difficulty.
众所周知，猜测会影响多项选择题测试的可靠性。尽管有许多研究考察了猜测的影响，但它们使用的假设相当严格（例如，平行测试假设、项目间相关性均匀、项目难度均匀以及各项目猜测水平均匀）来评估猜测与测试可靠性之间的关系。基于项目反应理论（IRT）框架，本研究调查了在更现实的条件下，猜测对可靠性的影响程度，在这些条件下，项目难度、项目区分度和猜测水平实际上会因三种不同的测试长度（TL）而在各项目间有所不同。通过同时考虑多个项目特征，本研究还专注于检验猜测与模拟中输入的其他变量之间的交互作用，以使结果更符合实际。通过用IRT模型参数来表示经典测试理论（CTT）项目统计量(α)系数和可靠性，便于对更现实的条件进行模拟以及计算可靠性和CTT项目统计量。除了猜测对可靠性的一般负面影响外，结果还显示了测试长度与猜测之间以及猜测与测试难度之间的交互作用。
