An artificial intelligence perspective on geriatric syndromes: assessing the information accuracy and readability of ChatGPT.

PURPOSE

ChatGPT, a comprehensive language processing model, provides the opportunity for supportive and professional interactions with patients. However, its use to address patients' frequently asked questions (FAQs) and the readability of the text generated by ChatGPT remain unexplored, particularly in geriatrics. We identified the FAQs about common geriatric syndromes and assessed the accuracy and readability of the responses provided by ChatGPT.

METHODS

Two geriatricians with extensive knowledge and experience in geriatric syndromes independently reviewed the 28 responses provided by ChatGPT. The accuracy of the responses generated by ChatGPT was categorized on a rating scale from 0 (harmful) to 4 (excellent) based on current guidelines and approaches. The readability of the text generated by ChatGPT was assessed by administering two tests: the Flesch-Kincaid Reading Ease (FKRE) and the Flesch-Kincaid Grade Level (FKGL).

RESULTS

ChatGPT-generated responses with an overall mean accuracy score of 88% (3.52/4). Responses generated for sarcopenia diagnosis and depression treatment in older adults had the lowest accuracy scores (2.0 and 2.5, respectively). The mean FKRE score of the texts was 25.2, while the mean FKGL score was 14.5.

CONCLUSION

The accuracy scores of the responses generated by ChatGPT were high in most common geriatric syndromes except for sarcopenia diagnosis and depression treatment. Moreover, the text generated by ChatGPT was very difficult to read and best understood by college graduates. ChatGPT may reduce the uncertainty many patients face. Nevertheless, it remains advisable to consult with subject matter experts when undertaking consequential decision-making.