Wilby Kyle John, El Hajj Maguy S, El-Bashir Marwa, Mraiche Fatima
College of Pharmacy, Qatar University, PO Box 2713, Doha, Qatar.
Curr Pharm Teach Learn. 2018 Apr;10(4):423-426. doi: 10.1016/j.cptl.2017.12.015. Epub 2018 Jan 17.
Written assessments are essential components of higher education practices. However, faculty members encounter common pitfalls when designing questions intended to evaluate student-learning outcomes. The objective of this project was to determine the impact of a mandatory examination peer review process on question accuracy, alignment with learning objectives, use of best practices in question design, and language/grammar.
A mandatory peer review process was implemented for all midterm (before phase) and final (after phase) examinations. Peer review occurred by two reviewers and followed a pre-defined guidance document. Non-punitive feedback given to faculty members served as the intervention. Frequencies of flagged questions according to guidance categories were compared between phases.
A total of 21 midterm and 21 final exam reviews were included in the analysis. A total of 637 questions were reviewed across all midterms and 1003 questions were reviewed across all finals. Few questions were flagged for accuracy and alignment with learning outcomes. The median total proportion of questions flagged for best practices was significantly lower for final exams versus midterm exams (15.8 vs. 6.45%, p = 0.014). The intervention did not influence language and grammar errors (9.68 vs. 10.0% of questions flagged before and after, respectively, p = 0.305).
A non-punitive peer review process for written examinations can overcome pitfalls in exam creation and improve best practices in question writing. The peer-review process had a substantial effect at flagging language/grammar errors but error rate did not differ between midterm and final exams.
书面评估是高等教育实践的重要组成部分。然而，教师在设计旨在评估学生学习成果的问题时会遇到常见的陷阱。本项目的目的是确定强制性考试同行评审过程对问题准确性、与学习目标的一致性、问题设计中最佳实践的运用以及语言/语法的影响。
对所有期中考试（阶段前）和期末考试（阶段后）实施了强制性同行评审过程。由两名评审员进行同行评审，并遵循一份预先定义的指导文件。向教师提供的非惩罚性反馈作为干预措施。比较各阶段根据指导类别标记的问题频率。
分析共纳入21次期中考试和21次期末考试的评审。所有期中考试共评审了637个问题，所有期末考试共评审了1003个问题。很少有问题因准确性和与学习成果的一致性而被标记。期末考试中因最佳实践被标记问题的总比例中位数显著低于期中考试（15.8%对6.45%，p = 0.014）。干预措施对语言和语法错误没有影响（前后分别有9.68%和10.0%的问题被标记，p = 0.305）。
书面考试的非惩罚性同行评审过程可以克服考试命题中的陷阱，并改进问题编写中的最佳实践。同行评审过程在标记语言/语法错误方面有显著效果，但期中考试和期末考试的错误率没有差异。
