Trustworthiness, Value, Danger, and Readability of ChatGPT-Generated Responses to Health Questions Related to Pulmonary Arterial Hypertension.

AIM

To enhance outcomes for patients with pulmonary arterial hypertension (PAH), comprehensive and individualized therapy is needed. A large language model called Generative Pre-trained Transformer (ChatGPT) has the ability to provide expert yet patient-friendly care. We wanted to determine how well ChatGPT could accurately and consistently respond to inquiries on knowledge and management for PAH.

MATERIALS AND METHODS

When 20 PAH patients were diagnosed, they were asked what concerns they had about PAH and what they had researched online. In the evaluation, it was determined that patients frequently searched the Internet for answers to eight queries. These eight queries were posed to ChatGPT, and their responses were recorded. Ten experts in the field of PAH assessed the trustworthiness, value, and hazard of the answers generated by the ChatGPT.

RESULTS

According to evaluations conducted by experts, the ChatGPT-generated responses were deemed trustworthy with an average score of 8.4 (7.7-9.2) and valuable with an average score of 7.9 (7.4-8.2). Based on the statistical analysis, it can be inferred that most professionals believed that the utilization of prompts provided by ChatGPT did not present a substantial risk, with a mean of 2.1 (1.7-2.5). The answers were assessed for readability using two different indicators, namely the Flesch-Kincaid Grade Level (FKGL) and the Simple Measure of Gobbledygook (SMOG). The average FKGL value was determined to be 13.52 ± 2.40, indicating a "difficult" level of readability.

CONCLUSION

ChatGPT provides reliable PAH-related information, but it is important to seek professional medical advice before making any decisions regarding PAH. ChatGPT can only provide general information and support, but a qualified healthcare provider can offer tailored recommendations.