Mershon J Patrick, Posid Tasha, Salari Keyan, Matulewicz Richard S, Singer Eric A, Dason Shawn
Division of Urologic Oncology, The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA.
Department of Urology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.
Transl Cancer Res. 2024 Nov 30;13(11):6246-6254. doi: 10.21037/tcr-23-2234. Epub 2024 May 21.
BACKGROUND: OpenAI's ChatGPT is a large language model-based artificial intelligence (AI) chatbot that can be used to answer unique, user-generated questions without direct training on specific content. Large language models have significant potential in urologic education. We reviewed the primary data surrounding the use of large language models in urology. We also reported findings of our primary study assessing the performance of ChatGPT in renal cell carcinoma (RCC) education. METHODS: For our primary study, we utilized three professional society guidelines addressing RCC to generate fifteen content questions. These questions were inputted into ChatGPT 3.5. ChatGPT responses along with pre- and post-content assessment questions regarding ChatGPT were then presented to evaluators. Evaluators consisted of four urologic oncologists and four non-clinical staff members. Medline was reviewed for additional studies pertaining to the use of ChatGPT in urologic education. RESULTS: We found that all assessors rated ChatGPT highly on the accuracy and usefulness of information provided with overall mean scores of 3.64 [±0.62 standard deviation (SD)] and 3.58 (±0.75) out of 5, respectively. Clinicians and non-clinicians did not differ in their scoring of responses (P=0.37). Completing content assessment improved confidence in the accuracy of ChatGPT's information (P=0.01) and increased agreement that it should be used for medical education (P=0.007). Attitudes towards use for patient education did not change (P=0.30). We also review the current state of the literature regarding ChatGPT use for patient and trainee education and discuss future steps towards optimization. CONCLUSIONS: ChatGPT has significant potential utility in medical education if it can continue to provide accurate and useful information. We have found it to be a useful adjunct to expert human guidance both for medical trainee and, less so, for patient education. Further work is needed to validate ChatGPT before widespread adoption.
背景:OpenAI的ChatGPT是一种基于大语言模型的人工智能(AI)聊天机器人,可用于回答用户提出的独特问题,而无需针对特定内容进行直接训练。大语言模型在泌尿外科教育中具有巨大潜力。我们回顾了围绕大语言模型在泌尿外科应用的主要数据。我们还报告了我们的初步研究结果,该研究评估了ChatGPT在肾细胞癌(RCC)教育中的表现。 方法:在我们的初步研究中,我们利用了三项关于RCC的专业学会指南来生成15个内容问题。这些问题被输入到ChatGPT 3.5中。然后将ChatGPT的回答以及关于ChatGPT的内容前和内容后评估问题呈现给评估人员。评估人员包括四名泌尿外科肿瘤学家和四名非临床工作人员。对Medline进行了检索,以查找有关ChatGPT在泌尿外科教育中应用的其他研究。 结果:我们发现,所有评估人员对ChatGPT提供信息的准确性和有用性评价都很高,总体平均得分分别为3.64[±0.62标准差(SD)]和3.58(±0.75)(满分5分)。临床医生和非临床医生对回答的评分没有差异(P=0.37)。完成内容评估提高了对ChatGPT信息准确性的信心(P=0.01),并增加了对其应用于医学教育的认可度(P=0.007)。对其用于患者教育的态度没有改变(P=0.30)。我们还回顾了关于ChatGPT用于患者和学员教育的文献现状,并讨论了未来优化的步骤。 结论:如果ChatGPT能够继续提供准确和有用的信息,它在医学教育中具有巨大的潜在效用。我们发现它对于医学实习生以及在较小程度上对于患者教育而言,是专家人工指导的有用辅助工具。在广泛采用之前,需要进一步开展工作来验证ChatGPT。
Front Med (Lausanne). 2023-12-13
Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2025-6-25
Curr Opin Urol. 2025-5-1
JMIR Med Educ. 2023-11-10