Marrow Scythia, Michaud Eric J, Hoel Erik
Allen Discovery Center, Tufts University, Medford, MA 02155, USA.
Department of Mathematics, University of California Berkeley, Berkeley, CA 94720, USA.
Entropy (Basel). 2020 Dec 18;22(12):1429. doi: 10.3390/e22121429.
Deep Neural Networks (DNNs) are often examined at the level of their response to input, such as analyzing the mutual information between nodes and data sets. Yet DNNs can also be examined at the level of causation, exploring "what does what" within the layers of the network itself. Historically, analyzing the causal structure of DNNs has received less attention than understanding their responses to input. Yet definitionally, generalizability must be a function of a DNN's causal structure as it reflects how the DNN responds to unseen or even not-yet-defined future inputs. Here, we introduce a suite of metrics based on information theory to quantify and track changes in the causal structure of DNNs during training. Specifically, we introduce the () of a feedforward DNN, which is the mutual information between layer input and output following a maximum-entropy perturbation. The can be used to assess the degree of causal influence nodes and edges have over their downstream targets in each layer. We show that the can be further decomposed in order to examine the sensitivity of a layer (measured by how well edges transmit perturbations) and the degeneracy of a layer (measured by how edge overlap interferes with transmission), along with estimates of the amount of integrated information of a layer. Together, these properties define where each layer lies in the "causal plane", which can be used to visualize how layer connectivity becomes more sensitive or degenerate over time, and how integration changes during training, revealing how the layer-by-layer causal structure differentiates. These results may help in understanding the generalization capabilities of DNNs and provide foundational tools for making DNNs both more generalizable and more explainable.
深度神经网络（DNN）通常在其对输入的响应层面进行研究，例如分析节点与数据集之间的互信息。然而，DNN也可以在因果关系层面进行研究，探索网络自身各层内部的“什么导致了什么”。从历史上看，分析DNN的因果结构比理解它们对输入的响应受到的关注更少。然而，从定义上讲，通用性必须是DNN因果结构的一个函数，因为它反映了DNN对未见过甚至尚未定义的未来输入的响应方式。在这里，我们引入了一套基于信息论的指标，用于量化和跟踪训练期间DNN因果结构的变化。具体来说，我们引入了前馈DNN的（），它是最大熵扰动后层输入与输出之间的互信息。该（）可用于评估各层中节点和边对其下游目标的因果影响程度。我们表明，该（）可以进一步分解，以便检查一层的敏感性（通过边传输扰动的程度来衡量）和一层的简并性（通过边的重叠对传输的干扰程度来衡量），以及一层的整合信息量估计。这些属性共同定义了每一层在“因果平面”中的位置，可用于可视化层连接性如何随时间变得更敏感或更简并，以及训练期间整合如何变化，揭示逐层因果结构是如何分化的。这些结果可能有助于理解DNN的泛化能力，并为使DNN更具泛化性和更具可解释性提供基础工具。
