Birks Samuel, Gray James, Darling-Pomranz Claire
School of Medicine and Population Health, University of Sheffield, Sheffield, UK.
Med Teach. 2025 Aug;47(8):1377-1384. doi: 10.1080/0142159X.2024.2434101. Epub 2024 Dec 1.
Generative AI can potentially streamline the creation of practice exam questions. This study sought to evaluate medical students' confidence using generative AI for this purpose, and overall attitudes towards its use.
The study used a mixed-methods approach with a pre-post intervention design. 68 medical and physician associate students were recruited to attend a workshop where they were shown how to use Google Bard (now Gemini) to write exam questions before being encouraged to do this themselves with guidance. A survey was completed before and after. Seven students also participated in a follow-up focus group.
The results showed an increase in participants' confidence in using AI to write practice exam questions ( < 0.001) after the workshop. Qualitative feedback highlighted pros and cons of using generative AI to write exam questions, alongside some concerns about its implementation. Students noted other positive uses in the curriculum and expressed a desire for institutional clarity on appropriate AI use.
While increased confidence is positive, rigorous evaluation of AI-generated question quality is needed to confirm accuracy. Teaching students to use generative AI to create and critique practice questions represents a means of encouraging appropriate AI use.
生成式人工智能有可能简化实践考试问题的创建。本研究旨在评估医学生使用生成式人工智能实现这一目的的信心，以及对其使用的总体态度。
本研究采用混合方法，采用干预前后设计。招募了68名医学生和医师助理学生参加一个工作坊，在工作坊中向他们展示如何使用谷歌巴德（现Gemini）编写考试问题，然后在指导下鼓励他们自己编写。前后各完成一次调查。七名学生还参加了后续焦点小组。
结果显示，工作坊结束后，参与者使用人工智能编写实践考试问题的信心有所增强（<0.001）。定性反馈突出了使用生成式人工智能编写考试问题的优缺点，以及对其实施的一些担忧。学生们指出了在课程中的其他积极用途，并表示希望机构在人工智能的适当使用方面明确规定。
虽然信心增强是积极的，但需要对人工智能生成的问题质量进行严格评估以确认准确性。教学生使用生成式人工智能创建和评判实践问题是鼓励适当使用人工智能的一种方式。
