Saad Muhammad, Moqeet Muhammad A, Mansoor Hassan, Khan Shama, Sharif Rabia, Khan Fahim Ullah, Naqvi Ali H, Ali Warda
Ophthalmology, Al-Shifa Trust Eye Hospital, Rawalpindi, PAK.
Cornea and Refractive Surgery, Al-Shifa Trust Eye Hospital, Rawalpindi, PAK.
Cureus. 2025 Feb 26;17(2):e79688. doi: 10.7759/cureus.79688. eCollection 2025 Feb.
Background Vernal keratoconjunctivitis (VKC) is a recurrent allergic eye disease that requires accurate patient education to ensure proper management. AI-driven chatbots, such as Google Gemini Advanced (Mountain View, California, US), are increasingly being explored as potential tools for providing medical information. This study evaluates the accuracy, reliability, and clinical applicability of Google Gemini Advanced in addressing VKC-related queries. Objective To assess the performance of Google Gemini Advanced in delivering medically accurate and relevant information about VKC and to evaluate its reliability based on expert ratings. Methods A total of 125 responses generated by Google Gemini Advanced for 25 VKC-related questions were assessed by two independent cornea specialists. Responses were rated on accuracy, completeness, and potential harm using a 5-point Likert scale (1-5). Inter-rater reliability was measured using Cronbach's alpha. Responses were categorized into highly accurate (score of 5), minor inconsistencies (score of 4), and inaccurate (scores 1-3). Results Google Gemini Advanced demonstrated high inter-rater reliability (Cronbach's alpha = 0.92, 95% CI: 0.87-0.94). Of the 125 responses, 108 (86.4%) were rated highly accurate (score of 5) while 17 (13.6%) had minor inconsistencies (score of 4) but posed no potential for harm. No responses were classified as inaccurate or potentially harmful. The combined mean score was 4.88 ± 0.31, reflecting strong agreement between raters. The chatbot consistently provided reliable information across diagnostic, treatment, and prognosis-related queries, with minor gaps in complex grading and treatment-related discussions. Discussion The findings support the use of AI-driven chatbots like Google Gemini Advanced as potential tools for patient education in ophthalmology. The chatbot exhibited strong accuracy and consistency, particularly in addressing general VKC-related queries. However, areas for improvement remain, especially in providing detailed guidance on treatment protocols and ensuring completeness in responses to complex clinical questions. Conclusion Google Gemini Advanced demonstrates high reliability and accuracy in delivering medical information about VKC, making it a valuable tool for patient education. While its responses are consistent and generally accurate, expert oversight remains necessary to refine AI-generated content for clinical applications. Further research is needed to enhance AI-driven chatbots' ability to provide nuanced medical advice and integrate them safely into ophthalmic patient education and clinical decision-making.
春季角结膜炎(VKC)是一种复发性过敏性眼病,需要对患者进行准确的教育以确保妥善管理。诸如谷歌Gemini Advanced(美国加利福尼亚州山景城)之类的人工智能驱动的聊天机器人正越来越多地被探索作为提供医疗信息的潜在工具。本研究评估了谷歌Gemini Advanced在回答VKC相关问题方面的准确性、可靠性和临床适用性。
评估谷歌Gemini Advanced在提供有关VKC的医学准确且相关信息方面的表现,并根据专家评分评估其可靠性。
由谷歌Gemini Advanced针对25个VKC相关问题生成的总共125个回答由两名独立的角膜专家进行评估。使用5点李克特量表(1 - 5)对回答的准确性、完整性和潜在危害进行评分。使用克朗巴哈系数测量评分者间信度。回答被分为高度准确(得分5)、轻微不一致(得分4)和不准确(得分1 - 3)。
谷歌Gemini Advanced显示出较高的评分者间信度(克朗巴哈系数 = 0.92,95%置信区间:0.87 - 0.94)。在125个回答中,108个(86.4%)被评为高度准确(得分5),而17个(13.6%)有轻微不一致(得分4)但没有潜在危害。没有回答被归类为不准确或有潜在危害。综合平均得分为4.88 ± 0.31,反映出评分者之间的高度一致性。该聊天机器人在诊断、治疗和预后相关问题上始终提供可靠信息,在复杂分级和治疗相关讨论方面存在一些小差距。
研究结果支持将诸如谷歌Gemini Advanced之类的人工智能驱动的聊天机器人作为眼科患者教育的潜在工具。该聊天机器人表现出很强的准确性和一致性,特别是在回答一般VKC相关问题方面。然而,仍有改进的空间,特别是在提供治疗方案的详细指导以及确保对复杂临床问题的回答完整方面。
谷歌Gemini Advanced在提供有关VKC的医学信息方面显示出高可靠性和准确性,使其成为患者教育的有价值工具。虽然其回答一致且总体准确,但仍需要专家监督以完善人工智能生成的内容用于临床应用。需要进一步研究以提高人工智能驱动的聊天机器人提供细致入微的医学建议的能力,并将它们安全地整合到眼科患者教育和临床决策中。