Department of Gastroenterology, Ganzhou People's Hospital, Jiangxi Medical College, Nanchang University, Nanchang, China.
Department of Gastroenterology, Shanghai Changhai Hospital, Naval Medical University, Shanghai, China.
Helicobacter. 2024 May-Jun;29(3):e13078. doi: 10.1111/hel.13078.
Educational initiatives on Helicobacter pylori (H. pylori) constitute a highly effective approach for preventing its infection and establishing standardized protocols for its eradication. ChatGPT, a large language model, is a potentially patient-friendly online tool capable of providing health-related knowledge. This study aims to assess the accuracy and repeatability of ChatGPT in responding to questions related to H. pylori.
Twenty-one common questions about H. pylori were collected and categorized into four domains: basic knowledge, diagnosis, treatment, and prevention. ChatGPT was utilized to individually answer the aforementioned 21 questions. Its responses were independently assessed by two experts on H. pylori. Questions with divergent ratings were resolved by a third reviewer. Cohen's kappa coefficient was calculated to assess the consistency between the scores of the two reviewers.
The responses of ChatGPT on H. pylori-related questions were generally satisfactory, with 61.9% marked as "completely correct" and 33.33% as "correct but inadequate." The repeatability of the responses of ChatGPT to H. pylori-related questions was 95.23%. Among the responses, those related to prevention (comprehensive: 75%) had the best response, followed by those on treatment (comprehensive: 66.7%), basic knowledge (comprehensive: 60%), and diagnosis (comprehensive: 50%). In the "treatment" domain, 16.6% of the ChatGPT responses were categorized as "mixed with correct or incorrect/outdated data." However, ChatGPT still lacks relevant knowledge regarding H. pylori resistance and the use of sensitive antibiotics.
ChatGPT can provide correct answers to the majority of H. pylori-related queries. It exhibited good reproducibility and delivered responses that were easily comprehensible to patients. Further enhancement of real-time information updates and correction of inaccurate information will make ChatGPT an essential auxiliary tool for providing accurate H. pylori-related health information to patients.
关于幽门螺杆菌（H. pylori）的教育计划是预防其感染和建立标准化根除方案的一种非常有效的方法。ChatGPT 是一种大型语言模型，是一种具有潜在患者友好性的在线工具，能够提供健康相关知识。本研究旨在评估 ChatGPT 在回答与 H. pylori 相关问题时的准确性和可重复性。
收集了 21 个关于 H. pylori 的常见问题，并分为四个领域：基础知识、诊断、治疗和预防。利用 ChatGPT 分别回答上述 21 个问题。由两位 H. pylori 专家对其回答进行独立评估。对评分存在分歧的问题，由第三位审稿人进行裁决。计算 Cohen's kappa 系数以评估两位审稿人评分的一致性。
ChatGPT 对 H. pylori 相关问题的回答总体上令人满意，61.9%标记为“完全正确”，33.33%标记为“正确但不充分”。ChatGPT 对 H. pylori 相关问题的回答的可重复性为 95.23%。在回答中，与预防相关的问题（全面：75%）回答最好，其次是治疗（全面：66.7%）、基础知识（全面：60%）和诊断（全面：50%）。在“治疗”领域，16.6%的 ChatGPT 回答被归类为“包含正确或错误/过时的数据”。然而，ChatGPT 仍然缺乏有关 H. pylori 耐药性和敏感抗生素使用的相关知识。
ChatGPT 可以为大多数 H. pylori 相关问题提供正确答案。它表现出良好的重现性，并为患者提供易于理解的回答。进一步增强实时信息更新和纠正不准确信息的能力，将使 ChatGPT 成为向患者提供准确的 H. pylori 相关健康信息的重要辅助工具。
