Andreou Vasiliki, Peters Sanne, Eggermont Jan, Wens Johan, Schoenmakers Birgitte
Department of Public Health and Primacy Care, KU Leuven, Academic Center for General Practice, Kapucijnenvoer 7 -Box 7001, 3000, Leuven, Belgium.
Evidence Based Practice, EBMPracticeNet, 3000, Leuven, Belgium.
BMC Med Educ. 2021 Dec 20;21(1):624. doi: 10.1186/s12909-021-03068-x.
The COVID-19 pandemic has profoundly affected assessment practices in medical education necessitating distancing from the traditional classroom. However, safeguarding academic integrity is of particular importance for high-stakes medical exams. We utilised remote proctoring to administer safely and reliably a proficiency-test for admission to the Advanced Master of General Practice (AMGP). We compared exam results of the remote proctored exam group to those of the on-site proctored exam group.
A cross-sectional design was adopted with candidates applying for admission to the AMGP. We developed and applied a proctoring software operating on three levels to register suspicious events: recording actions, analysing behaviour, and live supervision. We performed a Mann-Whitney U test to compare exam results from the remote proctored to the on-site proctored group. To get more insight into candidates' perceptions about proctoring, a post-test questionnaire was administered. An exploratory factor analysis was performed to explore quantitative data, while qualitative data were thematically analysed.
In total, 472 (79%) candidates took the proficiency-test using the proctoring software, while 121 (20%) were on-site with live supervision. The results indicated that the proctoring type does not influence exam results. Out of 472 candidates, 304 filled in the post-test questionnaire. Two factors were extracted from the analysis and identified as candidates' appreciation of proctoring and as emotional distress because of proctoring. Four themes were identified in the thematic analysis providing more insight on candidates' emotional well-being.
A comparison of exam results revealed that remote proctoring could be a viable solution for administering high-stakes medical exams. With regards to candidates' educational experience, remote proctoring was met with mixed feelings. Potential privacy issues and increased test anxiety should be taken into consideration when choosing a proctoring protocol. Future research should explore generalizability of these results utilising other proctoring systems in medical education and in other educational settings.
新冠疫情对医学教育中的评估实践产生了深远影响，使得必须与传统课堂保持距离。然而，对于高风险的医学考试而言，维护学术诚信尤为重要。我们利用远程监考安全可靠地进行了全科医学高级硕士（AMGP）入学能力测试。我们将远程监考考试组的考试结果与现场监考考试组的结果进行了比较。
采用横断面设计，研究对象为申请AMGP入学的考生。我们开发并应用了一个在三个层面运行的监考软件，以记录可疑事件：记录行为、分析行为和实时监督。我们进行了曼-惠特尼U检验，以比较远程监考组和现场监考组的考试结果。为了更深入了解考生对监考的看法，我们在考试后发放了问卷。对定量数据进行探索性因素分析，对定性数据进行主题分析。
共有472名（79%）考生使用监考软件参加了能力测试，121名（20%）考生在现场接受实时监督。结果表明，监考类型不影响考试成绩。在472名考生中，有304名填写了考试后问卷。从分析中提取了两个因素，分别被确定为考生对监考的认可和因监考产生的情绪困扰。在主题分析中确定了四个主题，能更深入地了解考生的情绪状况。
考试结果比较显示，远程监考可能是进行高风险医学考试的可行解决方案。关于考生的教育体验，考生对远程监考的感受不一。选择监考方案时应考虑潜在的隐私问题和考试焦虑增加的情况。未来的研究应探讨这些结果在医学教育及其他教育环境中使用其他监考系统时的可推广性。
