Islamaj Rezarta, Lai Po-Ting, Wei Chih-Hsuan, Luo Ling, Almeida Tiago, Jonker Richard A A, Conceição Sofia I R, Sousa Diana F, Phan Cong-Phuoc, Chiang Jung-Hsien, Li Jiru, Pan Dinghao, Meesawad Wilailack, Tsai Richard Tzong-Han, Sarol M Janina, Hong Gibong, Valiev Airat, Tutubalina Elena, Lee Shao-Man, Hsu Yi-Yu, Li Mingjie, Verspoor Karin, Lu Zhiyong
National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), 8600 Rockville Pike, Bethesda, MD 20894, United States.
School of Computer Science and Technology, Dalian University of Technology, No. 2 Linggong Road, Ganjingzi District, Dalian 116024, China.
Database (Oxford). 2024 Aug 8;2024. doi: 10.1093/database/baae069.
The BioRED track at BioCreative VIII calls for a community effort to identify, semantically categorize, and highlight the novelty factor of the relationships between biomedical entities in unstructured text. Relation extraction is crucial for many biomedical natural language processing (NLP) applications, from drug discovery to custom medical solutions. The BioRED track simulates a real-world application of biomedical relationship extraction, and as such, considers multiple biomedical entity types, normalized to their specific corresponding database identifiers, as well as defines relationships between them in the documents. The challenge consisted of two subtasks: (i) in Subtask 1, participants were given the article text and human expert annotated entities, and were asked to extract the relation pairs, identify their semantic type and the novelty factor, and (ii) in Subtask 2, participants were given only the article text, and were asked to build an end-to-end system that could identify and categorize the relationships and their novelty. We received a total of 94 submissions from 14 teams worldwide. The highest F-score performances achieved for the Subtask 1 were: 77.17% for relation pair identification, 58.95% for relation type identification, 59.22% for novelty identification, and 44.55% when evaluating all of the above aspects of the comprehensive relation extraction. The highest F-score performances achieved for the Subtask 2 were: 55.84% for relation pair, 43.03% for relation type, 42.74% for novelty, and 32.75% for comprehensive relation extraction. The entire BioRED track dataset and other challenge materials are available at https://ftp.ncbi.nlm.nih.gov/pub/lu/BC8-BioRED-track/ and https://codalab.lisn.upsaclay.fr/competitions/13377 and https://codalab.lisn.upsaclay.fr/competitions/13378. Database URL: https://ftp.ncbi.nlm.nih.gov/pub/lu/BC8-BioRED-track/https://codalab.lisn.upsaclay.fr/competitions/13377https://codalab.lisn.upsaclay.fr/competitions/13378.
生物创意竞赛 VIII 的生物关系抽取（BioRED）赛道呼吁社区共同努力，在非结构化文本中识别生物医学实体之间的关系、进行语义分类并突出其新颖性因素。关系抽取对于许多生物医学自然语言处理（NLP）应用至关重要，从药物发现到定制医疗解决方案。BioRED 赛道模拟了生物医学关系抽取的实际应用，因此考虑了多种生物医学实体类型，并将其标准化为特定的相应数据库标识符，同时还定义了文档中它们之间的关系。该挑战赛包括两个子任务：（i）在子任务 1 中，为参与者提供文章文本和人工专家标注的实体，要求他们提取关系对，识别其语义类型和新颖性因素；（ii）在子任务 2 中，仅向参与者提供文章文本，要求他们构建一个端到端系统，能够识别和分类关系及其新颖性。我们共收到来自全球 14 个团队的 94 份提交作品。子任务 1 取得最高 F 值的成绩分别为：关系对识别为 77.17%，关系类型识别为 58.95%，新颖性识别为 59.22%，在评估综合关系抽取的上述所有方面时为 44.55%。子任务 2 取得最高 F 值的成绩分别为：关系对为 55.84%，关系类型为 43.03%，新颖性为 42.74%，综合关系抽取为 32.75%。整个 BioRED 赛道数据集和其他挑战材料可在 https://ftp.ncbi.nlm.nih.gov/pub/lu/BC8-BioRED-track/、https://codalab.lisn.upsaclay.fr/competitions/13377 和 https://codalab.lisn.upsaclay.fr/competitions/1337８ 上获取。数据库网址：https://ftp.ncbi.nlm.nih.gov/pub/lu/BC8-BioRED-track/https://codalab.lisn.upsaclay.fr/competitions/13377https://codalab.lisn.upsaclay.fr/competitions/13378 。
