Evaluating ChatGPT platform in delivering heart failure educational material: A comparison with the leading national cardiology institutes.

BACKGROUND

Patient education plays a crucial role in improving the quality of life for patients with heart failure. As artificial intelligence continues to advance, new chatbots are emerging as valuable tools across various aspects of life. One prominent example is ChatGPT, a widely used chatbot among the public. Our study aims to evaluate the readability of ChatGPT answers for common patients' questions about heart failure.

METHODS

We performed a comparative analysis between ChatGPT responses and existing heart failure educational materials from top US cardiology institutes. Validated readability calculators were employed to assess and compare the reading difficulty and grade level of the materials. Furthermore, blind assessment using The Patient Education Materials Assessment Tool (PEMAT) was done by four advanced heart failure attendings to evaluate the readability and actionability of each resource.

RESULTS

Our study revealed that responses generated by ChatGPT were longer and more challenging to read compared to other materials. Additionally, these responses were written at a higher educational level (undergraduate and 9-10th grade), similar to those from the Heart Failure Society of America. Despite achieving a competitive PEMAT readability score (75 %), surpassing the American Heart Association score (68 %), ChatGPT's actionability score was the lowest (66.7 %) among all materials included in our study.

CONCLUSION

Despite its current limitations, artificial intelligence chatbots has the potential to revolutionize the field of patient education especially given theirs ongoing improvements. However, further research is necessary to ensure the integrity and reliability of these chatbots before endorsing them as reliable resources for patient education.