Demirbaş Kaan Can, Saygılı Seha, Yılmaz Esra Karabağ, Gülmez Rüveyda, Ağbaş Ayşe, Taşdemir Mehmet, Canpolat Nur
Department of Pediatrics, Istanbul University-Cerrahpaşa, Cerrahpaşa School of Medicine, Istanbul, Türkiye.
Division of Pediatric Nephrology, Department of Pediatrics, Istanbul University-Cerrahpaşa, Cerrahpaşa School of Medicine, Istanbul, Türkiye.
Pediatr Transplant. 2025 May;29(3):e70068. doi: 10.1111/petr.70068.
Education and enhancing the knowledge of adolescents who will undergo kidney transplantation are among the primary objectives of their care. While there are specific interventions in place to achieve this, they require extensive resources. The rise of large language models like ChatGPT-3.5 offers potential assistance for providing information to patients. This study aimed to evaluate the accuracy, relevance, and safety of ChatGPT-3.5's responses to patient-centered questions about pediatric kidney transplantation. The objective was to assess whether ChatGPT-3.5 could be a supplementary educational tool for adolescents and their caregivers in a complex medical context.
A total of 37 questions about kidney transplantation were presented to ChatGPT-3.5, which was prompted to respond as a health professional would to a layperson. Five pediatric nephrologists independently evaluated the outputs for accuracy, relevance, comprehensiveness, understandability, readability, and safety.
The mean accuracy, relevancy, and comprehensiveness scores for all outputs were 4.51, 4.56, and 4.55, respectively. Out of 37 outputs, four were rated as completely accurate, and seven were completely relevant and comprehensive. Only one output had an accuracy, relevancy, and comprehensiveness score below 4. Twelve outputs were considered potentially risky, but only three had a risk grade of moderate or higher. Outputs that were considered risky had an accuracy and relevancy below the average.
Our findings suggest that ChatGPT could be a useful tool for adolescents or caregivers of individuals waiting for kidney transplantation. However, the presence of potentially risky outputs underscores the necessity for human oversight and validation.
教育并增进即将接受肾移植的青少年的知识是其护理的主要目标之一。虽然有具体干预措施来实现这一目标,但需要大量资源。像ChatGPT-3.5这样的大语言模型的兴起为向患者提供信息提供了潜在帮助。本研究旨在评估ChatGPT-3.5对以患者为中心的小儿肾移植问题的回答的准确性、相关性和安全性。目的是评估ChatGPT-3.5在复杂医疗环境中是否可以成为青少年及其护理人员的辅助教育工具。
总共向ChatGPT-3.5提出了37个关于肾移植的问题,并要求其像健康专业人员对非专业人员那样进行回答。五位儿科肾脏病专家独立评估这些回答的准确性、相关性、全面性、可理解性、可读性和安全性。
所有回答 的平均准确性、相关性和全面性得分分别为4.51、4.56和4.55。在37个回答中,4个被评为完全准确,7个完全相关且全面。只有1个回答的准确性、相关性和全面性得分低于4分。12个回答被认为有潜在风险,但只有3个风险等级为中等或更高。被认为有风险的回答的准确性和相关性低于平均水平。
我们的研究结果表明,ChatGPT对于等待肾移植的青少年或其护理人员可能是一个有用的工具。然而,存在潜在风险的回答凸显了人工监督和验证的必要性。