Xiromeriti Maria, Newton Philip M
Swansea University Medical School, Swansea University, Singleton Park Campus, Swansea, Wales, SA2 8PP UK.
Med Sci Educ. 2024 Aug 20;34(6):1469-1477. doi: 10.1007/s40670-024-02140-7. eCollection 2024 Dec.
Problem-solving and higher-order learning are goals of higher education. It has been repeatedly suggested that multiple-choice questions (MCQs) can be used to test higher-order learning, although objective empirical evidence is lacking and MCQs are often criticised for assessing only lower-order, factual, or 'rote' learning. These challenges are compounded by a lack of agreement on what constitutes higher order learning: it is normally defined subjectively using heavily criticised frameworks such as such as Bloom's taxonomy. There is also a lack of agreement on how to write MCQs which assess higher order learning. Here we tested guidance for the creation of MCQs to assess higher-order learning, by evaluating the performance of students who were subject matter novices, vs experts. We found that questions written using the guidance were much harder to answer when students had no prior subject knowledge, whereas lower-order questions could be answered by simply searching online. These findings suggest that questions written using the guidance do indeed test higher-order learning, and such MCQs may be a valid alternative to other written assessment formats designed to test higher-order learning, such as essays, where reliability and cheating are a major concern.
The online version contains supplementary material available at 10.1007/s40670-024-02140-7.
解决问题和高阶学习是高等教育的目标。尽管缺乏客观的实证证据，且多项选择题（MCQ）常因仅能评估低阶、事实性或“死记硬背”的学习而受到批评，但人们多次建议可使用多项选择题来测试高阶学习。由于对于什么构成高阶学习缺乏共识，这些挑战变得更加复杂：通常使用诸如布鲁姆分类法等受到严厉批评的框架进行主观定义。对于如何编写评估高阶学习的多项选择题也缺乏共识。在此，我们通过评估学科新手与专家学生的表现，测试了用于创建评估高阶学习的多项选择题的指导方法。我们发现，当学生没有先前的学科知识时，按照该指导编写的问题更难回答，而低阶问题通过简单的在线搜索即可回答。这些发现表明，按照该指导编写的问题确实能测试高阶学习，并且此类多项选择题可能是用于测试高阶学习的其他书面评估形式（如论文，其可靠性和作弊是主要问题）的有效替代方式。
在线版本包含可在10.1007/s40670-024-02140-7获取的补充材料。
