Department of Medical Informatics, Erasmus University Medical Center, P.O. Box 2040, Rotterdam, CA 3000, The Netherlands.
Department of Human Genetics, Leiden University Medical Center, P.O. Box 9600, Leiden, RC 2300, The Netherlands.
J Cheminform. 2015 Jan 19;7(Suppl 1 Text mining for chemistry and the CHEMDNER track):S10. doi: 10.1186/1758-2946-7-S1-S10. eCollection 2015.
The past decade has seen an upsurge in the number of publications in chemistry. The ever-swelling volume of available documents makes it increasingly hard to extract relevant new information from such unstructured texts. The BioCreative CHEMDNER challenge invites the development of systems for the automatic recognition of chemicals in text (CEM task) and for ranking the recognized compounds at the document level (CDI task). We investigated an ensemble approach where dictionary-based named entity recognition is used along with grammar-based recognizers to extract compounds from text. We assessed the performance of ten different commercial and publicly available lexical resources using an open source indexing system (Peregrine), in combination with three different chemical compound recognizers and a set of regular expressions to recognize chemical database identifiers. The effect of different stop-word lists, case-sensitivity matching, and use of chunking information was also investigated. We focused on lexical resources that provide chemical structure information. To rank the different compounds found in a text, we used a term confidence score based on the normalized ratio of the term frequencies in chemical and non-chemical journals.
The use of stop-word lists greatly improved the performance of the dictionary-based recognition, but there was no additional benefit from using chunking information. A combination of ChEBI and HMDB as lexical resources, the LeadMine tool for grammar-based recognition, and the regular expressions, outperformed any of the individual systems. On the test set, the F-scores were 77.8% (recall 71.2%, precision 85.8%) for the CEM task and 77.6% (recall 71.7%, precision 84.6%) for the CDI task. Missed terms were mainly due to tokenization issues, poor recognition of formulas, and term conjunctions.
We developed an ensemble system that combines dictionary-based and grammar-based approaches for chemical named entity recognition, outperforming any of the individual systems that we considered. The system is able to provide structure information for most of the compounds that are found. Improved tokenization and better recognition of specific entity types is likely to further improve system performance.
过去十年，化学领域的文献数量呈爆炸式增长。可用文献的数量不断增加，使得从这些非结构化文本中提取相关新信息变得越来越困难。BioCreative CHEMDNER 挑战赛邀请开发系统，用于自动识别文本中的化学物质（CEM 任务）和对文档级别的识别化合物进行排名（CDI 任务）。我们研究了一种集成方法，该方法使用基于字典的命名实体识别，并结合基于语法的识别器从文本中提取化合物。我们使用开源索引系统（Peregrine）评估了十种不同的商业和公开可用的词汇资源的性能，同时结合了三种不同的化学化合物识别器和一组正则表达式来识别化学数据库标识符。还研究了不同停用词列表、大小写敏感匹配和使用分块信息的效果。我们专注于提供化学结构信息的词汇资源。为了对文本中找到的不同化合物进行排名，我们使用基于术语频率的归一化比的术语置信度得分，该比在化学和非化学期刊中。
使用停用词列表极大地提高了基于字典的识别性能，但使用分块信息没有额外的好处。ChEBI 和 HMDB 作为词汇资源的组合、基于语法的识别的 LeadMine 工具以及正则表达式，优于任何单个系统。在测试集上，CEM 任务的 F 分数为 77.8%（召回率 71.2%，精度 85.8%），CDI 任务的 F 分数为 77.6%（召回率 71.7%，精度 84.6%）。错过的术语主要是由于标记化问题、公式识别差和术语连接。
我们开发了一种集成系统，该系统结合了基于字典和语法的化学命名实体识别方法，优于我们考虑的任何单个系统。该系统能够为大多数找到的化合物提供结构信息。改进标记化和更好地识别特定实体类型可能会进一步提高系统性能。
