Pletscher-Frankild Sune, Jensen Lars Juhl
Disease Systems Biology Program, Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.
Intomics A/S, Lyngby, Denmark.
J Cheminform. 2019 Mar 8;11(1):19. doi: 10.1186/s13321-019-0344-9.
Most BioCreative tasks to date have focused on assessing the quality of text-mining annotations in terms of precision and recall. Interoperability, speed, and stability are, however, other important factors to consider for practical applications of text mining. For about a decade, we have run named entity recognition (NER) web services, which are designed to be efficient, implemented using a multi-threaded queueing system to robustly handle many simultaneous requests, and hosted at a supercomputer facility. To participate in this new task, we extended the existing NER tagging service with support for the BeCalm API. The tagger suffered no downtime during the challenge and, as in earlier tests, proved to be highly efficient, consistently processing requests of 5000 abstracts in less than half a minute. In fact, the majority of this time was spent not on the NER task but rather on retrieving the document texts from the challenge servers. The latter was found to be the main bottleneck even when hosting a copy of the tagging service on a Raspberry Pi 3, showing that local document storage or caching would be desirable features to include in future revisions of the API standard.
到目前为止，大多数生物创意任务都集中在根据精确率和召回率来评估文本挖掘注释的质量。然而，对于文本挖掘的实际应用而言，互操作性、速度和稳定性是其他需要考虑的重要因素。大约十年来，我们一直在运行命名实体识别（NER）网络服务，该服务设计高效，使用多线程排队系统实现，以稳健处理许多同时请求，并托管在超级计算机设施中。为了参与这项新任务，我们扩展了现有的NER标记服务，以支持BeCalm API。在挑战期间，标记器没有停机，并且和早期测试一样，被证明效率很高，始终能在不到半分钟的时间内处理5000篇摘要的请求。实际上，这段时间的大部分并非花在NER任务上，而是花在从挑战服务器检索文档文本上。即使在树莓派3上托管标记服务的副本时，后者也被发现是主要瓶颈，这表明本地文档存储或缓存将是未来API标准修订版中值得纳入的功能。
