Evaluation of AI chatbots for patient education and information on chronic obstructive pulmonary disease.

BACKGROUND

Chronic obstructive pulmonary disease (COPD) is a chronic and progressive disease that affects patients' quality of life and functional capacity. With its widespread use and ease of access, AI chatbots stand out as an alternative source of patient-centered information and education.

OBJECTIVES

To evaluate the readability and accuracy of information provided by ChatGPT, Gemini, and DeepSeek in COPD.

METHODS

Ten most frequently asked questions and answers regarding COPD in English were provided using three AI chatbots (ChatGPT-4 Turbo, Gemini 2.0 Flash, DeepSeek R1). Readability was assessed using the Flesch-Kincaid Grade Level (FKGL), while information quality was analyzed by five physiotherapists based on the guidelines. Responses were graded using a 4-point system from "excellent response requiring no explanation" to "unsatisfactory requiring significant explanation." Statistical analyses were performed on SPSS.

RESULTS

Overall, all three AI chatbots responded to questions with similar quality, with Gemini 2.0 providing a statistically higher quality response to question 4 (p < 0.05). In terms of readability of the answers, DeepSeek was found to have better readability on Q5 (12.01), Q8 (9.24), Q9 (13.1) and Q10 (8.73) compared to ChatGPT (Q5:13.9, Q8:11.92, Q9:17.15, Q10:9.88) and Gemini (Q5:18.22, Q8:15.47, Q9:17.42, Q10:9.38). Gemini was observed to produce more complex and academic level answers on more questions (Q4, Q5, Q8).

CONCLUSIONS

ChatGPT, Gemini, and DeepSeek provided evidence-based answers to frequently asked patient questions about COPD. DeepSeek showed better readability performance for many questions. AI chatbots may serve as a valuable clinical tool for COPD patient education and disease management in the future.