Łaszkiewicz Jan, Krajewski Wojciech, Tomczak Wojciech, Chorbińska Joanna, Nowak Łukasz, Chełmoński Adam, Krajewski Piotr, Sójka Aleksandra, Małkiewicz Bartosz, Szydełko Tomasz
University Center of Excellence in Urology, Wrocław Medical University, Wrocław, Poland.
Department of Minimally Invasive and Robotic Urology, University Center of Excellence in Urology, Wrocław Medical University, Wrocław, Poland.
Contemp Oncol (Pozn). 2024;28(2):172-181. doi: 10.5114/wo.2024.141567. Epub 2024 Aug 23.
The aim was to evaluate ChatGPT generated responses to patient-important questions regarding upper tract urothelial carcinoma (UTUC).
Fifteen common inquiries asked by patients regarding UTUC were assigned to 4 categories: general information; symptoms and diagnosis; treatment; and prognosis. These questions were entered into ChatGPT and its responses were recorded. In every answer 5 criteria (adequate length, comprehensible language, precision in addressing the question, compliance with European Association of Urology guidelines and safety of the response for the patient) were assessed by the urologists using a numerical scale of 1-5 (a score of 5 being the best).
Sixteen questionnaires were included. A score of five was assigned 336 times (28.0%); 4 - 527 times, (43.9%); 3 - 268 times (22.3%); 2 - 53 ti- mes (4.4%); and 1 - 16 times (1.3%). The average overall score was 3.93. Responses to each question received average scores within the range 3.34-4.18. Answers regarding "general information" were graded the highest - mean score 4.14. Artificial intelligence scored the lowest in the "treatment" category - mean score 3.68. A mean score of 4.02 was given for the safety of the response. However, a few urologists considered several answers as unsafe for the patient, by grading them 1 or 2 in this criterion.
ChatGPT does not provide fully adequate information on UTUC, and inquiries regarding treatment can be misleading for the patients. In particular cases, patients might receive potentially unsafe answers. However, ChatGPT can be used with caution to provide basic information regarding epidemiology and risk factors of UTUC.
目的是评估ChatGPT对有关上尿路尿路上皮癌（UTUC）的患者重要问题所生成的回答。
将患者关于UTUC提出的15个常见问题分为4类：一般信息；症状与诊断；治疗；以及预后。将这些问题输入ChatGPT并记录其回答。泌尿外科医生使用1至5的数字评分量表（5分为最佳）对每个回答的5项标准（回答长度合适、语言易懂、精准回答问题、符合欧洲泌尿外科协会指南以及回答对患者的安全性）进行评估。
纳入了16份问卷。5分被评定336次（28.0%）；4分 - 527次（43.9%）；3分 - 268次（22.3%）；2分 - 53次（4.4%）；以及1分 - 16次（1.3%）。平均总分是3.93。对每个问题的回答平均得分在3.34 - 4.18范围内。关于“一般信息”的回答评分最高 - 平均得分4.14。人工智能在“治疗”类别中得分最低 - 平均得分3.68。回答安全性的平均得分为4.02。然而，一些泌尿外科医生认为几个回答对患者不安全，在这一标准下给它们评分为1分或2分。
ChatGPT并未提供关于UTUC的充分信息，且关于治疗的询问可能会误导患者。在特定情况下，患者可能会收到潜在不安全的回答。然而，ChatGPT可谨慎用于提供有关UTUC流行病学和危险因素的基本信息。
