Ricci Leonardo
Dipartimento di Fisica, Università di Trento, 38123 Trento, Italy.
Phys Rev E. 2021 Feb;103(2-1):022215. doi: 10.1103/PhysRevE.103.022215.
The inference of Shannon entropy out of sample histograms is known to be affected by systematic and random errors that depend on the finite size of the available data set. This dependence was mostly investigated in the multinomial case, in which states are visited in an independent fashion. In this paper the asymptotic behavior of the distribution of the sample Shannon entropy, also referred to as plug-in estimator, is investigated in the case of an underlying finite Markov process characterized by a regular stochastic matrix. As the size of the data set tends to infinity, the plug-in estimator is shown to become asymptotically normal, though in a way that substantially deviates from the known multinomial case. The asymptotic behavior of bias and variance of the plug-in estimator are expressed in terms of the spectrum of the stochastic matrix and of the related covariance matrix. Effects of initial conditions are also considered. By virtue of the formal similarity with Shannon entropy, the results are directly applicable to the evaluation of permutation entropy.
已知从样本直方图推断香农熵会受到系统误差和随机误差的影响，这些误差取决于可用数据集的有限大小。这种依赖性大多是在多项式情形下进行研究的，在该情形中，各个状态是以独立方式出现的。在本文中，研究了样本香农熵（也称为插件估计器）分布的渐近行为，该研究针对的是由正则随机矩阵表征的潜在有限马尔可夫过程的情形。随着数据集大小趋于无穷大，插件估计器被证明会渐近正态，不过其方式与已知的多项式情形有很大偏差。插件估计器偏差和方差的渐近行为是根据随机矩阵的谱以及相关协方差矩阵来表示的。还考虑了初始条件的影响。由于与香农熵在形式上相似，这些结果可直接应用于排列熵的评估。
