Steger Diana, Schroeders Ulrich, Wilhelm Oliver
Ulm University, Ulm, Germany.
University of Kassel, Kassel, Germany.
Assessment. 2021 Apr;28(3):1004-1017. doi: 10.1177/1073191120914970. Epub 2020 May 1.
Cheating is a serious threat in unproctored ability assessment, irrespective of countermeasures taken, anticipated consequences (high vs. low stakes), and test modality (paper-pencil vs. computer-based). In the present study, we examined the power of (a) self-report-based indicators (i.e., Honesty-Humility and Overclaiming scales), (b) test data (i.e., performance with extremely difficult items), and (c) para data (i.e., reaction times, switching between browser tabs) to predict participants' cheating behavior. To this end, 315 participants worked on a knowledge test in an unproctored online assessment and subsequently in a proctored lab assessment. We used multiple regression analysis and an extended latent change score model to assess the potential of the different indicators to predict cheating. In summary, test data and para data performed best, while traditional self-report-based indicators were not predictive. We discuss the findings with respect to unproctored testing in general and provide practical advice on cheating detection in online ability assessments.
在无人监考的能力评估中,作弊是一个严重的威胁,无论采取何种对策、预期后果(高风险与低风险)以及测试方式(纸笔测试与计算机测试)如何。在本研究中,我们考察了以下因素预测参与者作弊行为的能力:(a) 基于自我报告的指标(即诚实-谦逊量表和过度宣称量表)、(b) 测试数据(即对极难项目的作答情况)以及 (c) 辅助数据(即反应时间、在浏览器标签之间切换的情况)。为此,315名参与者在无人监考的在线评估中进行了一次知识测试,随后又在有监考的实验室评估中进行了测试。我们使用多元回归分析和扩展的潜在变化分数模型来评估不同指标预测作弊行为的潜力。总之,测试数据和辅助数据表现最佳,而传统的基于自我报告的指标则没有预测能力。我们从总体上讨论了关于无人监考测试的研究结果,并就在线能力评估中的作弊检测提供了实用建议。