Moore Ian, Magnante Christopher, Embry Ellie, Mathis Jennifer, Mooney Scott, Haj-Hassan Shereen, Cottingham Maria, Padala Prasad R
Geriatric Research Education and Clinical Center (GRECC), Central Arkansas Veterans Healthcare System (CAVHS), Little Rock, AR, United States.
Tennessee Valley Veteran Affairs Healthcare System (TVHS), Nashville, TN, United States.
Front Artif Intell. 2024 Jul 25;7:1438012. doi: 10.3389/frai.2024.1438012. eCollection 2024.
AI technologies have the potential to transform patient care. AI has been used to aid in differential diagnosis and treatment planning for psychiatric disorders, administer therapeutic protocols, assist with interpretation of cognitive testing, and patient treatment planning. Despite advancements, AI has notable limitations and remains understudied and further research on its strengths and limitations in patient care is required. This study explored the responses of AI (Chat-GPT 3.5) and trained clinicians to commonly asked patient questions.
Three clinicians and AI provided responses to five dementia/geriatric healthcare-related questions. Responses were analyzed by a fourth, blinded clinician for clarity, accuracy, relevance, depth, and ease of understanding and to determine which response was AI generated.
AI responses were rated highest in ease of understanding and depth across all responses and tied for first for clarity, accuracy, and relevance. The rating for AI generated responses was 4.6/5 (SD = 0.26); the clinician s' responses were 4.3 (SD = 0.67), 4.2 (SD = 0.52), and 3.9 (SD = 0.59), respectively. The AI generated answers were identified in 4/5 instances.
AI responses were rated more highly and consistently on each question individually and overall than clinician answers demonstrating that AI could produce good responses to potential patient questions. However, AI responses were easily distinguishable from those of clinicians. Although AI has the potential to positively impact healthcare, concerns are raised regarding difficulties discerning AI from human generated material, the increased potential for proliferation of misinformation, data security concerns, and more.
人工智能技术有潜力改变患者护理。人工智能已被用于辅助精神疾病的鉴别诊断和治疗规划、执行治疗方案、协助认知测试解读以及患者治疗规划。尽管取得了进展，但人工智能存在显著局限性，仍未得到充分研究，需要进一步研究其在患者护理中的优势和局限性。本研究探讨了人工智能（Chat-GPT 3.5）和训练有素的临床医生对患者常见问题的回答。
三名临床医生和人工智能对五个与痴呆症/老年医疗保健相关的问题提供了回答。由第四名不知情的临床医生对回答进行分析，以评估清晰度、准确性、相关性、深度和易理解性，并确定哪些回答是由人工智能生成的。
在所有回答中，人工智能的回答在易理解性和深度方面得分最高，在清晰度、准确性和相关性方面并列第一。人工智能生成的回答评分是4.6/5（标准差 = 0.26）；临床医生的回答分别是4.3（标准差 = 0.67）、4.2（标准差 = 0.52）和3.9（标准差 = 0.59）。在五分之四的情况下能够识别出由人工智能生成的答案。
人工智能的回答在每个问题以及总体上的评分均高于临床医生的回答，这表明人工智能能够对患者可能提出的问题给出良好的回答。然而，人工智能的回答很容易与临床医生的回答区分开来。尽管人工智能有潜力对医疗保健产生积极影响，但人们对难以区分人工智能与人类生成的内容、错误信息传播增加的可能性、数据安全问题等表示担忧。
