Kleinman Michael, Achille Alessandro, Soatto Stefano, Kao Jonathan C
Department of Electrical and Computer Engineering, University of California, Los Angeles, CA 90095, USA.
Department of Computational and Mathematical Sciences, Caltech, Pasadena, CA 91125, USA.
Entropy (Basel). 2021 Jul 20;23(7):922. doi: 10.3390/e23070922.
We introduce the Redundant Information Neural Estimator (RINE), a method that allows efficient estimation for the component of information about a target variable that is common to a set of sources, known as the "redundant information". We show that existing definitions of the redundant information can be recast in terms of an optimization over a family of functions. In contrast to previous information decompositions, which can only be evaluated for discrete variables over small alphabets, we show that optimizing over functions enables the approximation of the redundant information for high-dimensional and continuous predictors. We demonstrate this on high-dimensional image classification and motor-neuroscience tasks.
我们介绍了冗余信息神经估计器（RINE），这是一种能够对一组源所共有的关于目标变量的信息成分（即“冗余信息”）进行有效估计的方法。我们表明，冗余信息的现有定义可以根据对一族函数的优化来重新表述。与之前只能针对小字母表上的离散变量进行评估的信息分解不同，我们表明对函数进行优化能够近似估计高维连续预测变量的冗余信息。我们在高维图像分类和运动神经科学任务中证明了这一点。
