Potential application of ChatGPT in disease relevant queries.

BACKGROUND

Advances in artificial intelligence are gradually transforming various fields, but its applicability among ordinary people is unknown. This study aims to explore the ability of a large language model to address related questions.

METHODS

We created several prompts on the basis of guidelines and the clinical concerns of patients. The capacity of ChatGPT on queries was evaluated by experts. Ordinary people assessed the applicability.

RESULTS

The responses to each prompt in ChatGPT-4 were good in terms of response length and repeatability. There was good agreement in each dimension (Fleiss' kappa ranged from 0.302 to 0.690,  < 0.05). The accuracy, completeness, usefulness, comprehension and satisfaction scores of the experts were generally high. Rated usefulness and comprehension among ordinary people were significantly lower than expert, while medical students gave a relatively positive evaluation.

CONCLUSION

ChatGPT-4 performs well in resolving related questions. Large language models may become an excellent tool for medical students in the future, but still requires further research and validation.