Gritta Milan, Pilehvar Mohammad Taher, Collier Nigel
Language Technology Lab (LTL), Department of Theoretical and Applied Linguistics (DTAL), University of Cambridge, 9 West Road, Cambridge, CB3 9DP UK.
Lang Resour Eval. 2020;54(3):683-712. doi: 10.1007/s10579-019-09475-3. Epub 2019 Sep 19.
Empirical methods in geoparsing have thus far lacked a standard evaluation framework describing the task, metrics and data used to compare state-of-the-art systems. Evaluation is further made inconsistent, even unrepresentative of real world usage by the lack of distinction between the , which necessitates new guidelines, a consolidation of metrics and a detailed toponym taxonomy with implications for Named Entity Recognition (NER) and beyond. To address these deficiencies, our manuscript introduces a new framework in three parts. (Part 1) Task Definition: clarified via corpus linguistic analysis proposing a fine-grained . (Part 2) Metrics: discussed and reviewed for a rigorous evaluation including recommendations for NER/Geoparsing practitioners. (Part 3) Evaluation data: shared via a new dataset called to provide test/train examples and enable immediate use of our contributions. In addition to fine-grained Geotagging and Toponym Resolution (Geocoding), this dataset is also suitable for prototyping and evaluating machine learning NLP models.
到目前为止，地理解析中的实证方法缺乏一个标准的评估框架来描述任务、指标以及用于比较最先进系统的数据。由于缺乏对[此处原文缺失部分内容]之间的区分，评估变得更加不一致，甚至不能代表现实世界的使用情况，这就需要新的指导方针、指标的整合以及一个详细的地名分类法，这对命名实体识别（NER）及其他方面都有影响。为了解决这些不足，我们的论文介绍了一个由三部分组成的新框架。（第一部分）任务定义：通过语料库语言学分析进行澄清，提出了一个细粒度的[此处原文缺失部分内容]。（第二部分）指标：进行了讨论和审查，以进行严格评估，包括为NER/地理解析从业者提供的建议。（第三部分）评估数据：通过一个名为[此处原文缺失数据集名称]的新数据集共享，以提供测试/训练示例，并使我们的贡献能够立即得到应用。除了细粒度的地理标记和地名解析（地理编码）外，这个数据集还适用于机器学习NLP模型的原型设计和评估。
