Departamento de Ciências da Saúde Pública e Forenses e Educação Médica, Unidade de Educação Médica, Faculdade de Medicina da Universidade do Porto, Piso 6, Al. Prof. Hernâni Monteiro, 4200 - 319, Porto, Portugal.
Institute of Public Health at the University of Porto, Porto, Portugal.
BMC Med Educ. 2019 Jul 25;19(1):284. doi: 10.1186/s12909-019-1710-z.
Computer and paper examinations in our days are constructed from an item pool which is regularly updated. Given the way that exams are created, one of the major concerns is the security of the items that are being used in order to ensure a good estimation of abilities. The aim of this study is to measure the prevalence of item pre-knowledge in our medical school.
The Deterministic, Gated Item Response Theory Model (DGM) was applied to estimate the prevalence of students who have had item pre-knowledge from six multiple choice examinations of the Clinical Anatomy course at the Faculty of Medicine of University of Porto. Each examination consisted of 100 items with an average of 200 students and 20% repeated items per examination. The estimation of the sensitivity and specificity was based on a simulation study. The sensitivity and specificity estimates, and apparent prevalence were used to estimate true prevalence of cheating students in the examinations under study.
The specificity in the DGM for different simulation scenarios was between 68 and 98%, while the sensitivity ranged from 60 to 91%. The apparent prevalence was between 0.0 and 3.4%, while the true prevalence ranged from 1.2 to 3.7%.
The true prevalence was much lower compared to the students self-reported copying of responses from other students; however, it is important to keep monitoring the pre-knowledge prevalence in order to enforce measures in case an increase occurs.
如今的计算机考试和纸质考试都是从定期更新的试题库中构建的。考虑到考试的编制方式，其中一个主要关注点是确保使用的试题具有安全性，以便能够对能力进行准确评估。本研究旨在衡量我们医学院学生中存在试题预先了解情况的普遍性。
应用确定性、有门限的项目反应理论模型（DGM）来估计波尔图大学医学院临床解剖学课程的六次多项选择考试中具有试题预先了解情况的学生的比例。每次考试包含 100 道试题，平均涉及 200 名学生，每次考试重复 20%的试题。基于模拟研究来估计敏感性和特异性。使用敏感性和特异性估计值以及明显的流行率来估计研究中的考试中学生作弊的真实流行率。
在不同模拟场景中，DGM 的特异性在 68%至 98%之间，而敏感性在 60%至 91%之间。明显的流行率在 0.0%至 3.4%之间，而真实的流行率在 1.2%至 3.7%之间。
与学生自我报告的从其他学生那里抄袭答案相比，真实的流行率要低得多；但是，为了在发生率增加时采取措施，有必要持续监测预先了解的流行率。
