Nielsen Frank
Sony Computer Science Laboratories, Tokyo 141-0022, Japan.
Entropy (Basel). 2024 Nov 22;26(12):1008. doi: 10.3390/e26121008.
The symmetric Kullback-Leibler centroid, also called the Jeffreys centroid, of a set of mutually absolutely continuous probability distributions on a measure space provides a notion of centrality which has proven useful in many tasks, including information retrieval, information fusion, and clustering. However, the Jeffreys centroid is not available in closed form for sets of categorical or multivariate normal distributions, two widely used statistical models, and thus needs to be approximated numerically in practice. In this paper, we first propose the new Jeffreys-Fisher-Rao center defined as the Fisher-Rao midpoint of the sided Kullback-Leibler centroids as a plug-in replacement of the Jeffreys centroid. This Jeffreys-Fisher-Rao center admits a generic formula for uni-parameter exponential family distributions and a closed-form formula for categorical and multivariate normal distributions; it matches exactly the Jeffreys centroid for same-mean normal distributions and is experimentally observed in practice to be close to the Jeffreys centroid. Second, we define a new type of inductive center generalizing the principle of the Gauss arithmetic-geometric double sequence mean for pairs of densities of any given exponential family. This new Gauss-Bregman center is shown experimentally to approximate very well the Jeffreys centroid and is suggested to be used as a replacement for the Jeffreys centroid when the Jeffreys-Fisher-Rao center is not available in closed form. Furthermore, this inductive center always converges and matches the Jeffreys centroid for sets of same-mean normal distributions. We report on our experiments, which first demonstrate how well the closed-form formula of the Jeffreys-Fisher-Rao center for categorical distributions approximates the costly numerical Jeffreys centroid, which relies on the Lambert function, and second show the fast convergence of the Gauss-Bregman double sequences, which can approximate closely the Jeffreys centroid when truncated to a first few iterations. Finally, we conclude this work by reinterpreting these fast proxy Jeffreys-Fisher-Rao and Gauss-Bregman centers of Jeffreys centroids under the lens of dually flat spaces in information geometry.
在一个测度空间上，一组相互绝对连续的概率分布的对称库尔贝克 - 莱布勒质心，也称为杰弗里斯质心，提供了一种中心性概念，已被证明在许多任务中很有用，包括信息检索、信息融合和聚类。然而，对于分类分布集或多元正态分布集（两种广泛使用的统计模型），杰弗里斯质心没有封闭形式，因此在实践中需要进行数值近似。在本文中，我们首先提出新的杰弗里斯 - 费希尔 - 拉奥中心，定义为有向库尔贝克 - 莱布勒质心的费希尔 - 拉奥中点，作为杰弗里斯质心的插件式替代。这个杰弗里斯 - 费希尔 - 拉奥中心对于单参数指数族分布有一个通用公式，对于分类分布和多元正态分布有一个封闭形式公式；对于均值相同的正态分布，它与杰弗里斯质心完全匹配，并且在实践中通过实验观察到它接近杰弗里斯质心。其次，我们定义了一种新型的归纳中心，它推广了任何给定指数族的密度对的高斯算术 - 几何双序列均值的原理。实验表明，这个新的高斯 - 布雷格曼中心能很好地近似杰弗里斯质心，并且当杰弗里斯 - 费希尔 - 拉奥中心没有封闭形式时，建议将其用作杰弗里斯质心的替代。此外，这个归纳中心总是收敛的，并且对于均值相同的正态分布集与杰弗里斯质心匹配。我们报告了我们的实验，首先展示了分类分布的杰弗里斯 - 费希尔 - 拉奥中心的封闭形式公式对依赖兰伯特函数的代价高昂的数值杰弗里斯质心的近似程度有多好，其次展示了高斯 - 布雷格曼双序列的快速收敛性，当截断到前几次迭代时，它可以紧密近似杰弗里斯质心。最后，我们通过在信息几何中的对偶平坦空间的视角下重新解释这些快速替代的杰弗里斯 - 费希尔 - 拉奥中心和杰弗里斯质心的高斯 - 布雷格曼中心来结束这项工作。
