Department of Medical Education and Research, Taipei Veterans General Hospital Hsinchu Branch, Hsinchu, Taiwan, ROC.
Department of Pharmacy, Taipei Veterans General Hospital Hsinchu Branch, Hsinchu, Taiwan, ROC.
J Chin Med Assoc. 2023 Jul 1;86(7):653-658. doi: 10.1097/JCMA.0000000000000942. Epub 2023 Jul 5.
ChatGPT is an artificial intelligence model trained for conversations. ChatGPT has been widely applied in general medical education and cardiology, but its application in pharmacy has been lacking. This study examined the accuracy of ChatGPT on the Taiwanese Pharmacist Licensing Examination and investigated its potential role in pharmacy education.
ChatGPT was used on the first Taiwanese Pharmacist Licensing Examination in 2023 in Mandarin and English. The questions were entered manually one by one. Graphical questions, chemical formulae, and tables were excluded. Textual questions were scored according to the number of correct answers. Chart question scores were determined by multiplying the number and the correct rate of text questions. This study was conducted from March 5 to March 10, 2023, by using ChatGPT 3.5.
The correct rate of ChatGPT in Chinese and English questions was 54.4% and 56.9% in the first stage, and 53.8% and 67.6% in the second stage. On the Chinese test, only pharmacology and pharmacochemistry sections received passing scores. The English test scores were higher than the Chinese test scores across all subjects and were significantly higher in dispensing pharmacy and clinical pharmacy as well as therapeutics.
ChatGPT 3.5 failed the Taiwanese Pharmacist Licensing Examination. Although it is not able to pass the examination, it can be improved quickly through deep learning. It reminds us that we should not only use multiple-choice questions to assess a pharmacist's ability, but also use more variety of evaluations in the future. Pharmacy education should be changed in line with the examination, and students must be able to use AI technology for self-learning. More importantly, we need to help students develop humanistic qualities and strengthen their ability to interact with patients, so that they can become warm-hearted healthcare professionals.
ChatGPT 是一款经过对话训练的人工智能模型。它已广泛应用于普通医学教育和心脏病学领域，但在药学领域的应用却相对较少。本研究旨在检验 ChatGPT 在台湾药师执照考试中的准确性，并探讨其在药学教育中的潜在作用。
2023 年，我们在首次以普通话和英语进行的台湾药师执照考试中使用了 ChatGPT。问题由人工逐个输入。排除图形问题、化学公式和表格。根据正确答案的数量给文本问题打分。图表问题的分数通过将文本问题的数量和正确率相乘来确定。本研究于 2023 年 3 月 5 日至 3 月 10 日进行，使用的是 ChatGPT 3.5。
ChatGPT 在第一阶段的中文和英文问题中的正确率分别为 54.4%和 56.9%，第二阶段分别为 53.8%和 67.6%。在中文考试中，只有药理学和药物化学部分达到及格分数。在所有科目中，英语考试的分数均高于中文考试，在配药和临床药学以及治疗学方面的分数差异尤为显著。
ChatGPT 3.5 未能通过台湾药师执照考试。虽然它还无法通过考试，但通过深度学习可以迅速提高它的表现。这提醒我们，我们不仅应该使用选择题来评估药师的能力，还应该在未来使用更多样化的评估方式。未来的药学教育应与考试相结合，学生必须能够使用 AI 技术进行自主学习。更重要的是，我们需要帮助学生培养人文素质，增强与患者互动的能力，使他们成为有温度的医疗保健专业人员。
