ChatGPT在韩国国家职业治疗师执照考试中的表现。
Performance of ChatGPT on the National Korean Occupational Therapy Licensing Examination.
作者信息
Lee Si-An, Heo Seoyoon, Park Jin-Hyuck
机构信息
Department of ICT convergence, The Graduate School, Soonchunhyang University, Asan, Republic of Korea.
Department of Occupational Therapy, Kyungbok University, Namyangju, Republic of Korea.
出版信息
Digit Health. 2024 Feb 29;10:20552076241236635. doi: 10.1177/20552076241236635. eCollection 2024 Jan-Dec.
BACKGROUND
ChatGPT is an artificial intelligence-based large language model (LLM). ChatGPT has been widely applied in medicine, but its application in occupational therapy has been lacking.
OBJECTIVE
This study examined the accuracy of ChatGPT on the National Korean Occupational Therapy Licensing Examination (NKOTLE) and investigated its potential for application in the field of occupational therapy.
METHODS
ChatGPT 3.5 was used during the five years of the NKOTLE with Korean prompts. Multiple choice questions were entered manually by three dependent encoders, and scored according to the number of correct answers.
RESULTS
During the most recent five years, ChatGPT did not achieve a passing score of 60% accuracy and exhibited interrater agreement of 0.6 or higher.
CONCLUSION
ChatGPT could not pass the NKOTLE but demonstrated a high level of agreement between raters. Even though the potential of ChatGPT to pass the NKOTLE is currently inadequate, it performed very close to the passing level even with only Korean prompts.
背景
ChatGPT是一种基于人工智能的大型语言模型（LLM）。ChatGPT已在医学中广泛应用，但其在职业治疗中的应用尚缺。
目的
本研究检验了ChatGPT在韩国国家职业治疗师执照考试（NKOTLE）中的准确性，并调查了其在职业治疗领域的应用潜力。
方法
在NKOTLE的五年考试期间使用ChatGPT 3.5并配以韩语提示。多项选择题由三名独立编码员手动输入，并根据正确答案数量评分。
结果
在最近五年中，ChatGPT未达到60%准确率的及格分数，且评分者间一致性为0.6或更高。
结论
ChatGPT未能通过NKOTLE，但在评分者间表现出高度一致性。尽管ChatGPT目前通过NKOTLE的潜力不足，但即使仅使用韩语提示，其表现也非常接近及格水平。
Zotero 插件