Ghosh Abhik, Basu Ayanendranath
Indian Statistical Institute, Kolkata 700108, India.
Entropy (Basel). 2018 May 6;20(5):347. doi: 10.3390/e20050347.
Entropy and relative entropy measures play a crucial role in mathematical information theory. The relative entropies are also widely used in statistics under the name of divergence measures which link these two fields of science through the minimum divergence principle. Divergence measures are popular among statisticians as many of the corresponding minimum divergence methods lead to robust inference in the presence of outliers in the observed data; examples include the ϕ -divergence, the density power divergence, the logarithmic density power divergence and the recently developed family of logarithmic super divergence (LSD). In this paper, we will present an alternative information theoretic formulation of the LSD measures as a two-parameter generalization of the relative α -entropy, which we refer to as the general ( α , β ) -entropy. We explore its relation with various other entropies and divergences, which also generates a two-parameter extension of Renyi entropy measure as a by-product. This paper is primarily focused on the geometric properties of the relative ( α , β ) -entropy or the LSD measures; we prove their continuity and convexity in both the arguments along with an extended Pythagorean relation under a power-transformation of the domain space. We also derive a set of sufficient conditions under which the forward and the reverse projections of the relative ( α , β ) -entropy exist and are unique. Finally, we briefly discuss the potential applications of the relative ( α , β ) -entropy or the LSD measures in statistical inference, in particular, for robust parameter estimation and hypothesis testing. Our results on the reverse projection of the relative ( α , β ) -entropy establish, for the first time, the existence and uniqueness of the minimum LSD estimators. Numerical illustrations are also provided for the problem of estimating the binomial parameter.
熵和相对熵度量在数学信息论中起着至关重要的作用。相对熵也以散度度量的名称在统计学中广泛使用，它通过最小散度原理将这两个科学领域联系起来。散度度量在统计学家中很受欢迎，因为许多相应的最小散度方法在观测数据存在异常值的情况下能导致稳健的推断；例子包括ϕ -散度、密度幂散度、对数密度幂散度以及最近发展起来的对数超散度（LSD）族。在本文中，我们将提出一种替代的信息论公式，将LSD度量表述为相对α -熵的双参数推广，我们将其称为广义(α, β ) -熵。我们探索它与各种其他熵和散度的关系，这也作为副产品产生了Renyi熵度量的双参数扩展。本文主要关注相对(α, β ) -熵或LSD度量的几何性质；我们证明了它们在两个自变量上的连续性和凸性，以及在定义域空间的幂变换下的扩展勾股关系。我们还推导了一组充分条件，在这些条件下相对(α, β ) -熵的正向和反向投影存在且唯一。最后，我们简要讨论相对(α, β ) -熵或LSD度量在统计推断中的潜在应用，特别是对于稳健参数估计和假设检验。我们关于相对(α, β ) -熵反向投影的结果首次确立了最小LSD估计量的存在性和唯一性。还提供了估计二项式参数问题的数值例证。
