Pu Jiaxi, Hong Jie, Yu Qiao, Yu Pan, Tian Jiaqi, He Yuehua, Huang Hanwei, Yuan Qiongjing, Tao Lijian, Peng Zhangzhe
Department of Nephrology, Xiangya Hospital, Central South University, Changsha, China.
Department of Nephrology, The Third Hospital of Changsha, Changsha, China.
Med Teach. 2025 Sep;47(9):1502-1508. doi: 10.1080/0142159X.2025.2458808. Epub 2025 Feb 2.
Recent advancements in artificial intelligence (AI) have enabled the customization of large language models to address specific domains such as medical education. This study investigates the practical performance of a custom GPT model in enhancing clinical knowledge acquisition for medical students and physicians.
A custom GPT was developed by incorporating the latest readily available teaching resources. Its accuracy in providing clinical knowledge was evaluated using a set of clinical questions, and responses were compared against established medical guidelines. Satisfaction was assessed through surveys involving medical students and physicians at different stages and from various types of hospitals. The impact of the custom GPT was further evaluated by comparing its role in facilitating clinical knowledge acquisition with traditional learning methods.
The custom GPT demonstrated higher accuracy (83.6%) compared to general AI models (65.5%, 69.1%) and was comparable to a professionally developed AI (Glass Health, 83.6%). Residents reported the highest satisfaction compared to clerks and physicians, citing improved learning independence, motivation, and confidence ( < 0.05). Physicians, especially those from teaching hospitals, showed greater eagerness to develop a custom GPT compared to clerks and residents ( < 0.05). The impact analysis revealed that residents using the custom GPT achieved better test scores compared to those using traditional resources ( < 0.05), though fewer perfect scores were obtained.
The custom GPT demonstrates significant promise as an innovative tool for advancing medical education, particularly for residents. Its capability to deliver accurate, tailored information complements traditional teaching methods, aiding educators in promoting personalized and consistent training. However, it is essential for both learners and educators to remain critical in evaluating AI-generated information. With continued development and thoughtful integration, AI tools like custom GPTs have the potential to significantly enhance the quality and accessibility of medical education.
