Choi Seul Ki, Moon Yunseo, Jung Hyunggu
Graduate School of Urban Public Health, University of Seoul, Seoul, Republic of Korea.
Department of Computer Science and Engineering, University of Seoul, Seoul, Republic of Korea.
Digit Health. 2025 Aug 21;11:20552076251361381. doi: 10.1177/20552076251361381. eCollection 2025 Jan-Dec.
ChatGPT's potential as a diet information tool is emerging. However, little is known about the extent to which the information provided by ChatGPT aligns with that provided by dietitians.
This study aimed to assess ChatGPT's capacity to provide responses to diet-related questions, compared to responses by dietitians.
A total of 928 diet-related questions and corresponding responses from dietitians were collected from Naver Knowledge-iN, a Korean online Q&A platform, between January 18, 2023, and January 17, 2024. ChatGPT-4o was used to generate responses to the same questions. Five text similarity indices-Dice Coefficient, Jaccard Index, Overlap Coefficient, Cosine Similarity, and Term Frequency-Inverse Document Frequency-were used to assess the similarity between ChatGPT's and dietitians' responses. Questions with the top 5% response similarity were reviewed to identify characteristics of the questions for which ChatGPT generated responses similar to those of dietitians. Responses with the bottom 5% similarity were reviewed to identify reasons for the low similarity.
The average similarity coefficient between ChatGPT and dietitian responses was 0.42. Questions with high response similarity tended to include detailed information, such as specific food items or portions (76.1%), the questioner's context (69.6%), or personal characteristics (17.4%). Low response similarity was mainly due to ChatGPT providing significantly longer responses than dietitians.
ChatGPT demonstrated content similarity to dietitian responses, but they were not identical. The development of prompt engineering techniques to enhance ChatGPT's ability to provide more expert-like and personalized information could benefit users seeking dietary information.
ChatGPT作为一种饮食信息工具的潜力正在显现。然而，对于ChatGPT提供的信息与营养师提供的信息的契合程度，人们知之甚少。
本研究旨在评估ChatGPT与营养师相比，对饮食相关问题提供回答的能力。
2023年1月18日至2024年1月17日期间，从韩国在线问答平台Naver Knowledge-iN收集了928个与饮食相关的问题及营养师的相应回答。使用ChatGPT-4o对相同问题生成回答。采用五种文本相似度指标——骰子系数、杰卡德指数、重叠系数、余弦相似度和词频逆文档频率——来评估ChatGPT与营养师回答之间的相似度。对回答相似度排名前5%的问题进行审查，以确定ChatGPT生成与营养师相似回答的问题特征。对相似度排名后5%的回答进行审查，以确定相似度低的原因。
ChatGPT与营养师回答之间的平均相似度系数为0.42。回答相似度高的问题往往包含详细信息，如特定食物项目或份量（76.1%）、提问者的背景（69.6%）或个人特征（17.4%）。回答相似度低主要是因为ChatGPT提供的回答比营养师的长得多。
ChatGPT显示出与营养师回答的内容相似度，但并不完全相同。开发提示工程技术以增强ChatGPT提供更专业和个性化信息的能力，可能会使寻求饮食信息的用户受益。
