Schoenmakers Birgitte, Wens Johan
Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium.
Department of Primary and Interdisciplinary Care, University of Antwerp, Antwerpen, Belgium.
JMIR Form Res. 2021 Aug 16;5(8):e23834. doi: 10.2196/23834.
The COVID-19 pandemic has affected education and assessment programs and has resulted in complex planning. Therefore, we organized the proficiency test for admission to the Family Medicine program as a proctored exam. To prevent fraud, we developed a web-based supervisor app for tracking and tracing candidates' behaviors.
We aimed to assess the efficiency and usability of the proctored exam procedure and to analyze the procedure's impact on exam scores.
The application operated on the following three levels to register events: the recording of actions, analyses of behavior, and live supervision. Each suspicious event was given a score. To assess efficiency, we logged the technical issues and the interventions. To test usability, we counted the number of suspicious students and behaviors. To analyze the impact that the supervisor app had on students' exam outcomes, we compared the scores of the proctored group and those of the on-campus group. Candidates were free to register for off-campus participation or on-campus participation.
Of the 593 candidates who subscribed to the exam, 472 (79.6%) used the supervisor app and 121 (20.4%) were on campus. The test results of both groups were comparable. We registered 15 technical issues that occurred off campus. Further, 2 candidates experienced a negative impact on their exams due to technical issues. The application detected 22 candidates with a suspicion rating of >1. Suspicion ratings mainly increased due to background noise. All events occurred without fraudulent intent.
This pilot observational study demonstrated that a supervisor app that records and registers behavior was able to detect suspicious events without having an impact on exams. Background noise was the most critical event. There was no fraud detected. A supervisor app that registers and records behavior to prevent fraud during exams was efficient and did not affect exam outcomes. In future research, a controlled study design should be used to compare the cost-benefit balance between the complex interventions of the supervisor app and candidates' awareness of being monitored via a safe browser plug-in for exams.
新冠疫情影响了教育和评估项目，导致规划工作变得复杂。因此，我们将家庭医学专业入学资格考试组织成一场监考考试。为防止作弊，我们开发了一款基于网络的监考应用程序，用于跟踪和追查考生的行为。
我们旨在评估监考考试程序的效率和可用性，并分析该程序对考试成绩的影响。
该应用程序在以下三个层面运行以记录事件：行为记录、行为分析和实时监督。每个可疑事件都被赋予一个分数。为评估效率，我们记录了技术问题和干预措施。为测试可用性，我们统计了可疑学生和行为的数量。为分析监考应用程序对学生考试结果的影响，我们比较了监考组和校内组的成绩。考生可自由选择报名参加校外考试或校内考试。
在报名参加考试的593名考生中，472名（79.6%）使用了监考应用程序，121名（20.4%）在校内参加考试。两组的测试结果具有可比性。我们记录了15起校外发生的技术问题；此外，有2名考生因技术问题考试受到负面影响。该应用程序检测到22名可疑评级大于1的考生。可疑考生。可疑评级主要因背景噪音而增加。所有事件均无欺诈意图。
这项初步观察性研究表明，一款记录和登记行为的监考应用程序能够检测到可疑事件，且不会对考试产生影响。背景噪音是最关键的事件。未发现作弊行为。一款用于在考试期间记录和登记行为以防止作弊的监考应用程序是有效的，且不影响考试结果。在未来的研究中，应采用对照研究设计，以比较监考应用程序的复杂干预措施与考生通过安全浏览器插件接受考试监控的意识之间的成本效益平衡。
