Geneva School of Business Administration (DMML Group), HES-SO, 1227 Geneva, Switzerland.
Sensors (Basel). 2022 Jan 30;22(3):1088. doi: 10.3390/s22031088.
Despite the great attention that the research community has paid to the creation of novel indoor positioning methods, a rather limited volume of works has focused on the confidence that Indoor Positioning Systems (IPS) assign to the position estimates that they produce. The concept of estimating, dynamically, the accuracy of the position estimates provided by an IPS has been sporadically studied in the literature of the field. Recently, this concept has started being studied as well in the context of outdoor positioning systems of Internet of Things (IoT) based on Low-Power Wide-Area Networks (LPWANs). What is problematic is that the consistent comparison of the proposed methods is quasi nonexistent: new methods rarely use previous ones as baselines; often, a small number of evaluation metrics are reported while different metrics are reported among different relevant publications, the use of open data is rare, and the publication of open code is absent. In this work, we present an open-source, reproducible benchmarking framework for evaluating and consistently comparing various methods of Dynamic Accuracy Estimation (DAE). This work reviews the relevant literature, presenting in a consistent terminology commonalities and differences and discussing baselines and evaluation metrics. Moreover, it evaluates multiple methods of DAE using open data, open code, and a rich set of relevant evaluation metrics. This is the first work aiming to establish the state of the art of methods of DAE determination in IPS and in LPWAN positioning systems, through an open, transparent, holistic, reproducible, and consistent evaluation of the methods proposed in the relevant literature.
尽管研究界非常关注新颖的室内定位方法的创建，但只有相当有限的一部分工作专注于室内定位系统 (IPS) 对其生成的位置估计赋予的置信度。在该领域的文献中，已经零星地研究了动态估计 IPS 提供的位置估计准确性的概念。最近，这个概念也开始在基于低功耗广域网 (LPWAN) 的物联网 (IoT) 室外定位系统的背景下进行研究。有问题的是，提出的方法之间的一致比较几乎不存在：新方法很少以前面的方法作为基准；通常，只报告少数几个评估指标，而不同的相关出版物则报告不同的指标，很少使用公开数据，也没有公开代码。在这项工作中，我们提出了一个开源的、可重复的基准框架，用于评估和一致比较各种动态准确性估计 (DAE) 方法。这项工作回顾了相关文献，以一致的术语呈现了共性和差异，并讨论了基准和评估指标。此外，它使用公开数据、公开代码和丰富的相关评估指标评估了多种 DAE 方法。这是第一项旨在通过对相关文献中提出的方法进行开放、透明、全面、可重复和一致的评估，来确定 IPS 和 LPWAN 定位系统中 DAE 确定方法的现状的工作。
