Using large language models to generate child-friendly education materials on myopia.

AIM

To evaluate the ability of large language models (LLMs) to produce patient education materials for myopic children and their parents.

METHODS

Thirty-five common myopia-related questions were used with two distinct prompts to produce responses aimed at adults (Prompt A) and children (Prompt B). Five ophthalmologists evaluated the responses using a 5-point Likert scale for correctness, completeness, conciseness, and potential harm. Readability was assessed via Flesch-Kincaid scores. The Kruskal-Wallis and Mann-Whitney tests were used to identify significant differences in LLM performance.

RESULTS

ChatGPT 4o achieved the most positive ratings ("good" and above) in correctness (Prompt A: 91%; Prompt B: 83%) and conciseness (Prompt A: 79%; Prompt B: 63%), as well as the lowest negative ratings in potential harm ratings ("not at all" and "slightly," Prompt A: 99%; Prompt B: 97%) in the generation of educational materials for both adults and children (all  < 0.001). In terms of completeness, the results varied between the two prompts. Specifically, in Prompt A, ChatGPT 4.0 demonstrated the highest level of completeness (ChatGPT 4o: 69%, ChatGPT 4.0: 74%, ChatGPT 3.5: 51%, Gemini: 73%,  < 0.001), whereas in Prompt B, ChatGPT 4o achieved the highest score (ChatGPT 4o: 71%, ChatGPT 4.0: 65%, ChatGPT 3.5: 38%, Gemini: 46%,  < 0.001). The responses generated with Prompt B were significantly more readable than those generated with Prompt A across all LLMs ( ≤ 0.001).

CONCLUSION

Large language models, particularly ChatGPT 4o, hold potential for delivering effective patient education materials on myopia for both adult and pediatric populations. While generally effective, LLMs have limitations for complex medical queries, necessitating continued refinement for reliable clinical use.