Singla Ria, Lodhi Sumiya, Kibret Taddele, Jegatheswaran Januvi, Glavinovic Tamara, Massicotte-Azarniouch David, Karpinski Jolanta, Powell Rinu, Burns Kevin, Sood Manish M, Bugeja Ann
Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
Clin Transplant. 2025 Sep;39(9):e70303. doi: 10.1111/ctr.70303.
The effectiveness of ChatGPT responses to common living kidney donation (LKD) queries remains unclear.
We surveyed nephrologists and living kidney donors/candidates to evaluate ChatGPT-3.5's accuracy, comprehensiveness, and clarity in answering common donation questions in English and French. Ratings used a 5-point Likert scale, with percentage agreement and modified Fleiss' Kappa measuring inter-rater consistency.
The evaluation of ChatGPT-3.5's responses varied between nephrologists and kidney donors/candidates. Nephrologists showed moderate percentage agreement for English responses (50%-59%) and poor agreement for French responses (9%-45%). Kidney donors/candidates exhibited high agreement for English (90%-100%) but low for French (0%-77%). Inter-rater agreement among nephrologists was moderate for both English (Kappa 0.74, 95% CI: 0.67, 0.79, p < 0.0001) and French (Kappa 0.70, 95% CI: 0.64, 0.77, p < 0.0001). In contrast, inter-rater agreement was poor among donors/candidates for both English (Kappa -0.10, 95% CI: -0.14, -0.07, p = 0.99) and French (Kappa -0.03, 95% CI: -0.07, 0, p = 0.81).
ChatGPT 3.5's responses to common LKD queries demonstrated limited agreement among nephrologists and kidney donors/donor candidates, highlighting its lack of reliability as a supplement to existing educational materials for living kidney donor programs in English and French.
ChatGPT对常见活体肾捐赠（LKD）问题的回答效果尚不清楚。
我们对肾病学家以及活体肾捐赠者/候选者进行了调查，以评估ChatGPT-3.5在以英语和法语回答常见捐赠问题时的准确性、全面性和清晰度。评分采用5分制李克特量表，用百分比一致性和修正的弗莱斯kappa系数来衡量评分者间的一致性。
肾病学家和肾捐赠者/候选者对ChatGPT-3.5回答的评价存在差异。肾病学家对英语回答的百分比一致性中等（50%-59%），对法语回答的一致性较差（9%-45%）。肾捐赠者/候选者对英语回答的一致性较高（90%-100%），但对法语回答的一致性较低（0%-77%）。肾病学家之间，英语（kappa系数0.74，95%置信区间：0.67，0.79，p<0.0001）和法语（kappa系数0.70，95%置信区间：0.64，0.77，p<0.0001）的评分者间一致性均为中等。相比之下，捐赠者/候选者之间，英语（kappa系数-0.10，95%置信区间：-0.14，-0.07，p=0.99）和法语（kappa系数-0.03，95%置信区间：-0.07，0，p=0.81）的评分者间一致性均较差。
ChatGPT 3.5对常见LKD问题的回答在肾病学家和肾捐赠者/候选者之间显示出有限的一致性，突出了其作为英语和法语活体肾捐赠项目现有教育材料补充的可靠性不足。
