Artificial intelligence with ChatGPT 4: a large language model in support of ocular oncology cases.

PURPOSE

To evaluate ChatGPT's ability to analyze comprehensive case descriptions of patients with uveal melanoma and provide recommendations for the most appropriate management.

DESIGN

Retrospective analysis of ocular oncology patients' medical records.

SUBJECTS

Forty patients treated for uveal melanoma between May 2019 and October 2023.

METHODS

We uploaded each case description into the ChatGPT interface (version 4.0) and asked the model to provide realistic treatment options by asking the question, "What type of treatment do you recommend?" The accuracy of decisions produced by ChatGPT was compared to those recorded in patients' files and the treatment recommendations provided by three ocular oncologists, each with more than 10 years of experience.

MAIN OUTCOME MEASURES

The primary objective of this research was to assess the accuracy of ChatGPT replies in ocular oncology cases, analyzing its competence in both straightforward and intricate situations. Our secondary purpose was to assess the concordance between the responses of ChatGPT and those of ocular oncology specialists when faced with analogous clinical scenarios.

RESULTS

ChatGPT's surgical choices matched those in patients' files in 55% of cases (22 out of 40). ChatGPT options were agreed upon by 50%, 55%, and 57% of the three ocular oncology specialists. The investigation revealed significant differences between ChatGPT's responses and those of the three cancer specialists when compared to patients' files (p = 0.003, p = 0.001, and p = 0.001). ChatGPT's surgical responses matched with patient data in 18 out of 24 cases (75%), excluding enucleation cases. The decisions matched with the three ocular oncology specialists in 17/24, 18/24, and 18/24 cases, reflecting agreements of 70%, 75%, and 75%, respectively. The decisions made by ChatGPT were not significantly different from those of the three professionals in this cohort (p = 0.50, p = 0.36, and p = 0.36 for ChatGPT compared to specialists 1, 2, and 3).

CONCLUSION

ChatGPT exhibited a level of proficiency that was comparable to that of trained ocular oncology specialists. However, it exhibited its limitations when evaluating more complex scenarios, such as extrascleral extension or infiltration of the optic nerve, when a comprehensive evaluation of the patient is therefore necessary.