Department of Educational Psychology, University of Illinois, Urbana-Champaign, Urbana-Champaign, Illinois, USA.
Department of Psychology and Statistics, University of Illinois, Urbana-Champaign, Urbana-Champaign, Illinois, USA.
Br J Math Stat Psychol. 2023 Feb;76(1):131-153. doi: 10.1111/bmsp.12286. Epub 2022 Sep 7.
Psychometric methods for accurate and timely detection of item compromise have been a long-standing topic. While Bayesian methods can incorporate prior knowledge or expert inputs as additional information for item compromise detection, they have not been employed in item compromise detection itself. The current study proposes a two-phase Bayesian change-point framework for both stationary and real-time detection of changes in each item's compromise status. In Phase I, a stationary Bayesian change-point model for compromise detection is fitted to the observed responses over a specified time-frame. The model produces parameter estimates for the change-point of each item from uncompromised to compromised, as well as structural parameters accounting for the post-change response distribution. Using the post-change model identified in Phase I, the Shiryaev procedure for sequential testing is employed in Phase II for real-time monitoring of item compromise. The proposed methods are evaluated in terms of parameter recovery, detection accuracy, and detection efficiency under various simulation conditions and in a real data example. The proposed method also showed superior detection accuracy and efficiency compared to the cumulative sum procedure.
准确、及时地检测项目受干扰的心理计量学方法一直是一个长期存在的话题。虽然贝叶斯方法可以将先验知识或专家输入作为附加信息纳入项目受干扰检测，但它们尚未应用于项目受干扰检测本身。本研究提出了一种两阶段贝叶斯变点框架，用于对每个项目的受干扰状态变化进行固定和实时检测。在第一阶段，拟合了一个固定贝叶斯变点模型来检测在特定时间段内的受干扰情况。该模型为每个项目从不受干扰到受干扰的变点以及考虑了变点后响应分布的结构参数产生了参数估计。使用第一阶段确定的变点模型，在第二阶段使用 Shiryaev 序贯检验程序对项目受干扰进行实时监测。在各种模拟条件下和一个真实数据示例中，对所提出的方法进行了参数恢复、检测准确性和检测效率方面的评估。与累积和程序相比，所提出的方法显示出了更高的检测准确性和效率。
