Özer Özcan Zeynep, Doğan Levent, Yilmaz Ibrahim Edhem
Department of Ophthalmology (Z.O.O.), Gaziantep State Hospital, Gaziantep, Turkey; Department of Ophthalmology (L.D.), Niğde Ömer Halisdemir University, Niğde, Turkey; and Department of Ophthalmology (I.E.Y.), Gaziantep Dr. Ersin Arslan Training and Research Hospital, Gaziantep, Turkey.
Eye Contact Lens. 2025 Mar 1;51(3):e112-e116. doi: 10.1097/ICL.0000000000001160. Epub 2024 Dec 31.
We aimed to compare the answers given by ChatGPT, Bard, and Copilot and that obtained from the American Academy of Ophthalmology (AAO) website to patient-written questions related to keratoconus in terms of accuracy, understandability, actionability, and readability to find out whether chatbots can be used in patient education.
Twenty patient-written questions obtained from the AAO website related to keratoconus were asked to ChatGPT, Bard, and Copilot. Two ophthalmologists independently assessed the answers obtained from chatbots and the AAO website in terms of accuracy, understandability, and actionability according to the Structure of Observed Learning Outcome taxonomy, Patient Education Materials Assessment Tool-Understandability, and Patient Education Materials Assessment Tool-Actionability tests, respectively. The answers were also compared for readability according to the Flesch Reading Ease scores obtained through the website.
Bard had significantly higher scores compared with ChatGPT-3.5, Copilot, and AAO website according to Structure of Observed Learning Outcome taxonomy and Patient Education Materials Assessment Tool-Understandability ( P <0.001 for each), whereas there was no significant difference between the other groups. Bard and ChatGPT achieved significantly higher scores than the AAO website according to the Patient Education Materials Assessment Tool-Actionability scale ( P =0.001). The AAO website achieved significantly higher scores than the Bard on the Flesch Reading Ease scale, whereas there was no significant difference between the other groups ( P =0.017).
Chatbots are promising to provide accurate, understandable, and actionable answers. Chatbots can be a valuable aid in the education of patients with keratoconus under clinician supervision. In this way, unnecessary hospital visits can be prevented, and the burden on the health care system can be alleviated, while patient awareness can be raised.
我们旨在比较ChatGPT、Bard和Copilot给出的答案与从美国眼科学会（AAO）网站获取的针对患者撰写的与圆锥角膜相关问题的答案，在准确性、可理解性、可操作性和可读性方面进行比较，以确定聊天机器人是否可用于患者教育。
向ChatGPT、Bard和Copilot提出从AAO网站获取的20个与圆锥角膜相关的患者撰写问题。两位眼科医生分别根据观察学习成果分类法、患者教育材料评估工具-可理解性和患者教育材料评估工具-可操作性测试，对从聊天机器人和AAO网站获得的答案在准确性、可理解性和可操作性方面进行独立评估。还根据通过网站获得的弗莱什阅读简易度分数对答案的可读性进行比较。
根据观察学习成果分类法和患者教育材料评估工具-可理解性，Bard的得分显著高于ChatGPT-3.5、Copilot和AAO网站（每项P<0.001），而其他组之间无显著差异。根据患者教育材料评估工具-可操作性量表，Bard和ChatGPT的得分显著高于AAO网站（P=0.001）。在弗莱什阅读简易度量表上，AAO网站的得分显著高于Bard，而其他组之间无显著差异（P=0.017）。
聊天机器人有望提供准确、可理解且可操作的答案。在临床医生的监督下，聊天机器人可为圆锥角膜患者的教育提供有价值的帮助。通过这种方式，可以避免不必要的医院就诊，减轻医疗保健系统的负担，同时提高患者的意识。
