Chen Chao, Twycross Jamie, Garibaldi Jonathan M
School of Computer Science, University of Nottingham, Nottingham, United Kingdom.
PLoS One. 2017 Mar 24;12(3):e0174202. doi: 10.1371/journal.pone.0174202. eCollection 2017.
Many accuracy measures have been proposed in the past for time series forecasting comparisons. However, many of these measures suffer from one or more issues such as poor resistance to outliers and scale dependence. In this paper, while summarising commonly used accuracy measures, a special review is made on the symmetric mean absolute percentage error. Moreover, a new accuracy measure called the Unscaled Mean Bounded Relative Absolute Error (UMBRAE), which combines the best features of various alternative measures, is proposed to address the common issues of existing measures. A comparative evaluation on the proposed and related measures has been made with both synthetic and real-world data. The results indicate that the proposed measure, with user selectable benchmark, performs as well as or better than other measures on selected criteria. Though it has been commonly accepted that there is no single best accuracy measure, we suggest that UMBRAE could be a good choice to evaluate forecasting methods, especially for cases where measures based on geometric mean of relative errors, such as the geometric mean relative absolute error, are preferred.
过去已经提出了许多用于时间序列预测比较的准确性度量。然而，这些度量中的许多都存在一个或多个问题，例如对异常值的抵抗力差和尺度依赖性。在本文中，在总结常用准确性度量的同时，对对称平均绝对百分比误差进行了特别回顾。此外，还提出了一种新的准确性度量，称为无标度平均有界相对绝对误差（UMBRAE），它结合了各种替代度量的最佳特性，以解决现有度量的常见问题。使用合成数据和真实世界数据对所提出的度量和相关度量进行了比较评估。结果表明，所提出的度量在用户可选择的基准下，在选定标准上的表现与其他度量相当或更好。尽管人们普遍认为没有单一的最佳准确性度量，但我们建议UMBRAE可能是评估预测方法的一个不错选择，特别是对于那些更喜欢基于相对误差几何平均值的度量（如几何平均相对绝对误差）的情况。
