Faculty of Science and Health, Charles Sturt University, Bathurst NSW, Australia.
UniSA Allied Health & Human Performance, University of South Australia, Adelaide, SA, Australia.
J Educ Eval Health Prof. 2024;21:34. doi: 10.3352/jeehp.2024.21.34. Epub 2024 Nov 18.
This study evaluates the use of ChatGPT-4o in creating tailored continuing professional development (CPD) plans for radiography students, addressing the challenge of aligning CPD with Medical Radiation Practice Board of Australia (MRPBA) requirements. We hypothesized that ChatGPT-4o could support students in CPD planning while meeting regulatory standards.
A descriptive, experimental design was used to generate 3 unique CPD plans using ChatGPT-4o, each tailored to hypothetical graduate radiographers in varied clinical settings. Each plan followed MRPBA guidelines, focusing on computed tomography specialization by the second year. Three MRPBA-registered academics assessed the plans using criteria of appropriateness, timeliness, relevance, reflection, and completeness from October 2024 to November 2024. Ratings underwent analysis using the Friedman test and intraclass correlation coefficient (ICC) to measure consistency among evaluators.
ChatGPT-4o generated CPD plans generally adhered to regulatory standards across scenarios. The Friedman test indicated no significant differences among raters (P=0.420, 0.761, and 0.807 for each scenario), suggesting consistent scores within scenarios. However, ICC values were low (-0.96, 0.41, and 0.058 for scenarios 1, 2, and 3), revealing variability among raters, particularly in timeliness and completeness criteria, suggesting limitations in the ChatGPT-4o's ability to address individualized and context-specific needs.
ChatGPT-4o demonstrates the potential to ease the cognitive demands of CPD planning, offering structured support in CPD development. However, human oversight remains essential to ensure plans are contextually relevant and deeply reflective. Future research should focus on enhancing artificial intelligence's personalization for CPD evaluation, highlighting ChatGPT-4o's potential and limitations as a tool in professional education.
本研究评估了 ChatGPT-4o 在为放射学学生制定定制的持续专业发展（CPD）计划方面的应用，旨在解决将 CPD 与澳大利亚医疗放射实践委员会（MRPBA）要求保持一致的挑战。我们假设 ChatGPT-4o 可以在满足监管标准的同时支持学生进行 CPD 规划。
采用描述性实验设计，使用 ChatGPT-4o 生成 3 个独特的 CPD 计划，每个计划都针对不同临床环境下的假设毕业生放射技师。每个计划都遵循 MRPBA 指南，重点是第二年的计算机断层扫描专业。三位注册为 MRPBA 的学者使用适当性、及时性、相关性、反思和完整性标准评估这些计划，评估时间为 2024 年 10 月至 2024 年 11 月。使用 Friedman 检验和组内相关系数（ICC）对评分进行分析，以衡量评估者之间的一致性。
ChatGPT-4o 生成的 CPD 计划在所有场景中总体上都符合监管标准。Friedman 检验表明，评分者之间没有显著差异（每个场景的 P=0.420、0.761 和 0.807），表明在每个场景内评分具有一致性。然而，ICC 值较低（场景 1、2 和 3 的值分别为-0.96、0.41 和 0.058），表明评分者之间存在差异，特别是在及时性和完整性标准方面，这表明 ChatGPT-4o 在满足个性化和具体情境需求方面存在局限性。
ChatGPT-4o 显示出减轻 CPD 规划认知负担的潜力，为 CPD 发展提供了结构化支持。然而，为了确保计划具有情境相关性和深度反思性，仍然需要人工监督。未来的研究应侧重于增强人工智能在 CPD 评估方面的个性化，突出 ChatGPT-4o 作为专业教育工具的潜力和局限性。
