Zhang Jeffrey, Wibert Maxwell, Zhou Huixue, Peng Xueqing, Chen Qingyu, Keloth Vipina K, Hu Yan, Zhang Rui, Xu Hua, Raja Kalpana
Section for Biomedical Informatics and Data Science, School of Medicine, Yale University, New Haven, USA.
Institute for Health Informatics, University of Minnesota, Twin Cities, USA.
AMIA Jt Summits Transl Sci Proc. 2024 May 31;2024:391-400. eCollection 2024.
Relation Extraction (RE) is a natural language processing (NLP) task for extracting semantic relations between biomedical entities. Recent developments in pre-trained large language models (LLM) motivated NLP researchers to use them for various NLP tasks. We investigated GPT-3.5-turbo and GPT-4 on extracting the relations from three standard datasets, EU-ADR, Gene Associations Database (GAD), and ChemProt. Unlike the existing approaches using datasets with masked entities, we used three versions for each dataset for our experiment: a version with masked entities, a second version with the original entities (unmasked), and a third version with abbreviations replaced with the original terms. We developed the prompts for various versions and used the chat completion model from GPT API. Our approach achieved a F1-score of 0.498 to 0.809 for GPT-3.5-turbo, and a highest F1-score of 0.84 for GPT-4. For certain experiments, the performance of GPT, BioBERT, and PubMedBERT are almost the same.
关系提取（RE）是一种用于提取生物医学实体之间语义关系的自然语言处理（NLP）任务。预训练大语言模型（LLM）的最新进展促使NLP研究人员将其用于各种NLP任务。我们研究了GPT-3.5-turbo和GPT-4从三个标准数据集（欧盟-药物不良反应（EU-ADR）、基因关联数据库（GAD）和化学蛋白质（ChemProt））中提取关系的能力。与现有的使用带有掩码实体的数据集的方法不同，我们在实验中为每个数据集使用了三个版本：一个带有掩码实体的版本、一个带有原始实体（未掩码）的第二个版本以及一个用原始术语替换缩写的第三个版本。我们为各种版本开发了提示，并使用了来自GPT API的聊天完成模型。我们的方法在GPT-3.5-turbo上的F1分数为0.498至0.809，GPT-4的最高F1分数为0.84。在某些实验中，GPT、BioBERT和PubMedBERT的性能几乎相同。
