Ariel Gil, Louzoun Yoram
Department of Mathematics, Bar Ilan University, Ramat Gan 5290002, Israel.
Entropy (Basel). 2020 Feb 19;22(2):236. doi: 10.3390/e22020236.
A method for estimating the Shannon differential entropy of multidimensional random variables using independent samples is described. The method is based on decomposing the distribution into a product of marginal distributions and joint dependency, also known as the copula. The entropy of marginals is estimated using one-dimensional methods. The entropy of the copula, which always has a compact support, is estimated recursively by splitting the data along statistically dependent dimensions. The method can be applied both for distributions with compact and non-compact supports, which is imperative when the support is not known or of a mixed type (in different dimensions). At high dimensions (larger than 20), numerical examples demonstrate that our method is not only more accurate, but also significantly more efficient than existing approaches.
描述了一种使用独立样本估计多维随机变量香农微分熵的方法。该方法基于将分布分解为边际分布和联合依赖性（也称为copula）的乘积。边际熵使用一维方法进行估计。copula的熵总是具有紧致支撑，通过沿统计相关维度分割数据进行递归估计。该方法可应用于具有紧致和非紧致支撑的分布，当支撑未知或为混合类型（在不同维度）时这一点至关重要。在高维度（大于20）时，数值示例表明我们的方法不仅更准确，而且比现有方法显著更高效。
