Ogrinc Matevž, Koroušić Seljak Barbara, Eftimov Tome
Jožef Stefan International Postgraduate School, Ljubljana, Slovenia.
Department of Computer Systems, Jožef Stefan Institute, Ljubljana, Slovenia.
Front Nutr. 2024 Aug 13;11:1429259. doi: 10.3389/fnut.2024.1429259. eCollection 2024.
Recognizing and extracting key information from textual data plays an important role in intelligent systems by maintaining up-to-date knowledge, reinforcing informed decision-making, question-answering, and more. It is especially apparent in the food domain, where critical information guides the decisions of nutritionists and clinicians. The information extraction process involves two natural language processing tasks named entity recognition-NER and named entity linking-NEL. With the emergence of large language models (LLMs), especially ChatGPT, many areas began incorporating its knowledge to reduce workloads or simplify tasks. In the field of food, however, we noticed an opportunity to involve ChatGPT in NER and NEL.
To assess ChatGPT's capabilities, we have evaluated its two versions, ChatGPT-3.5 and ChatGPT-4, focusing on their performance across both NER and NEL tasks, emphasizing food-related data. To benchmark our results in the food domain, we also investigated its capabilities in a more broadly investigated biomedical domain. By evaluating its zero-shot capabilities, we were able to ascertain the strengths and weaknesses of the two versions of ChatGPT.
Despite being able to show promising results in NER compared to other models. When tasked with linking entities to their identifiers from semantic models ChatGPT's effectiveness falls drastically.
While the integration of ChatGPT holds potential across various fields, it is crucial to approach its use with caution, particularly in relying on its responses for critical decisions in food and bio-medicine.
从文本数据中识别和提取关键信息在智能系统中起着重要作用，它能保持知识更新、加强明智决策、问答等。这在食品领域尤为明显，关键信息指导着营养学家和临床医生的决策。信息提取过程涉及两个自然语言处理任务，即命名实体识别（NER）和命名实体链接（NEL）。随着大语言模型（LLMs）的出现，尤其是ChatGPT，许多领域开始纳入其知识以减轻工作量或简化任务。然而，在食品领域，我们发现了让ChatGPT参与NER和NEL的机会。
为了评估ChatGPT的能力，我们评估了它的两个版本，ChatGPT-3.5和ChatGPT-4，重点关注它们在NER和NEL任务中的表现，强调与食品相关的数据。为了在食品领域对我们的结果进行基准测试，我们还研究了它在更广泛研究的生物医学领域的能力。通过评估其零样本能力，我们能够确定ChatGPT两个版本的优缺点。
尽管与其他模型相比，ChatGPT在NER方面能够显示出有希望的结果。但当任务是将实体与其语义模型中的标识符链接起来时，ChatGPT的有效性会大幅下降。
虽然ChatGPT的整合在各个领域都有潜力，但谨慎使用它至关重要，特别是在依靠其回答做出食品和生物医学关键决策时。
