Wyse Adam E, Babcock Ben
The American Registry of Radiologic Technologists, St. Paul, MN, USA.
Educ Psychol Meas. 2016 Jun;76(3):508-527. doi: 10.1177/0013164415588947. Epub 2015 Jun 1.
Continuously administered examination programs, particularly credentialing programs that require graduation from educational programs, often experience seasonality where distributions of examine ability may differ over time. Such seasonality may affect the quality of important statistical processes, such as item response theory (IRT) item calibration and equating. The lead time required for producing pre-equated test forms in the continuous testing framework further complicates issues. This study examines the effect of seasonality in test data on Rasch IRT item parameter estimates. Data came from four credentialing examination programs that represented both programs with and without seasonality, as well as medium and low examinee volume. Results showed that calibrating items during certain times can lead to quite poor item parameter estimates. While certain programs could conduct IRT calibrations without waiting for the full examination cycle to be completed, other types of programs should wait as long as possible before calibrating items.
持续实施的考试项目，尤其是那些要求从教育项目毕业才能获得资质的认证项目，常常会出现季节性现象，即考生能力分布可能随时间而有所不同。这种季节性可能会影响重要统计过程的质量，如项目反应理论（IRT）项目校准和等值化。在连续测试框架中生成预等值测试形式所需的提前期进一步使问题复杂化。本研究考察了测试数据中的季节性对Rasch IRT项目参数估计的影响。数据来自四个认证考试项目，这些项目既代表了有季节性和无季节性的项目，也代表了考生数量中等和较少的项目。结果表明，在特定时间校准项目可能会导致非常差的项目参数估计。虽然某些项目可以在不等待整个考试周期完成的情况下进行IRT校准，但其他类型的项目在项目校准前应尽可能长时间等待。
