Finn Conor, Lizier Joseph T
Complex Systems Research Group and Centre for Complex Systems, Faculty of Engineering & IT, The University of Sydney, NSW 2006, Australia.
CSIRO Data61, Marsfield NSW 2122, Australia.
Entropy (Basel). 2018 Apr 18;20(4):297. doi: 10.3390/e20040297.
What are the distinct ways in which a set of predictor variables can provide information about a target variable? When does a variable provide unique information, when do variables share redundant information, and when do variables combine synergistically to provide complementary information? The redundancy lattice from the partial information decomposition of Williams and Beer provided a promising glimpse at the answer to these questions. However, this structure was constructed using a much criticised measure of redundant information, and despite sustained research, no completely satisfactory replacement measure has been proposed. In this paper, we take a different approach, applying the axiomatic derivation of the redundancy lattice to a single realisation from a set of discrete variables. To overcome the difficulty associated with signed pointwise mutual information, we apply this decomposition separately to the unsigned entropic components of pointwise mutual information which we refer to as the specificity and ambiguity. This yields a separate redundancy lattice for each component. Then based upon an operational interpretation of redundancy, we define measures of redundant specificity and ambiguity enabling us to evaluate the partial information atoms in each lattice. These atoms can be recombined to yield the sought-after multivariate information decomposition. We apply this framework to canonical examples from the literature and discuss the results and the various properties of the decomposition. In particular, the pointwise decomposition using specificity and ambiguity satisfies a chain rule over target variables, which provides new insights into the so-called two-bit-copy example.
一组预测变量可以通过哪些不同方式提供关于目标变量的信息？变量何时提供独特信息，何时共享冗余信息，以及何时协同组合以提供互补信息？Williams和Beer的部分信息分解中的冗余格为这些问题的答案提供了有希望的线索。然而，这种结构是使用一种备受批评的冗余信息度量构建的，并且尽管进行了持续研究，但尚未提出完全令人满意的替代度量。在本文中，我们采用不同的方法，将冗余格的公理推导应用于一组离散变量的单个实现。为了克服与有符号逐点互信息相关的困难，我们将这种分解分别应用于逐点互信息的无符号熵分量，我们将其称为特异性和模糊性。这为每个分量产生一个单独的冗余格。然后基于对冗余的操作解释，我们定义冗余特异性和模糊性的度量，使我们能够评估每个格中的部分信息原子。这些原子可以重新组合以产生所需的多变量信息分解。我们将此框架应用于文献中的典型示例，并讨论结果以及分解的各种属性。特别是，使用特异性和模糊性的逐点分解满足目标变量上的链式法则，这为所谓的两位复制示例提供了新的见解。
