Evaluation of Information Provided by ChatGPT Versions on Traumatic Dental Injuries for Dental Students and Professionals.

BACKGROUND/AIM: The use of AI-driven chatbots for accessing medical information is increasingly popular among educators and students. This study aims to assess two different ChatGPT models-ChatGPT 3.5 and ChatGPT 4.0-regarding their responses to queries about traumatic dental injuries, specifically for dental students and professionals.

MATERIAL AND METHODS

A total of 40 questions were prepared, divided equally between those concerning definitions and diagnosis and those on treatment and follow-up. The responses from both ChatGPT versions were evaluated on several criteria: quality, reliability, similarity, and readability. These evaluations were conducted using the Global Quality Scale (GQS), the Reliability Scoring System (adapted DISCERN), the Flesch Reading Ease Score (FRES), the Flesch-Kincaid Reading Grade Level (FKRGL), and the Similarity Index. Normality was checked with the Shapiro-Wilk test, and variance homogeneity was assessed using the Levene test.

RESULTS

The analysis revealed that ChatGPT 3.5 provided more original responses compared to ChatGPT 4.0. According to FRES scores, both versions were challenging to read, with ChatGPT 3.5 having a higher FRES score (39.732 ± 9.713) than ChatGPT 4.0 (34.813 ± 9.356), indicating relatively better readability. There were no significant differences between the ChatGPT versions regarding GQS, DISCERN, and FKRGL scores. However, in the definition and diagnosis section, ChatGPT 4.0 had a statistically higher quality score than ChatGPT 3.5. In contrast, ChatGPT 3.5 provided more original answers in the treatment and follow-up section. For ChatGPT 4.0, the readability and similarity rates for the definition and diagnosis section were higher than those for the treatment and follow-up section. No significant differences were observed between ChatGPT 3.5's DISCERN, FRES, FKRGL, and similarity index measurements by topic.

CONCLUSIONS

Both ChatGPT versions offer high-quality and original information, though they present challenges in readability and reliability. They are valuable resources for dental students and professionals but should be used in conjunction with additional sources of information for a comprehensive understanding.