Theory Center, High Energy Accelerator Research Organization (KEK), Tsukuba, Ibaraki 305-0801, Japan.
Graduate University for Advanced Studies (SOKENDAI), Tsukuba, Ibaraki 305-0801, Japan.
Phys Rev E. 2018 May;97(5-1):053304. doi: 10.1103/PhysRevE.97.053304.
Theoretical understanding of how a deep neural network (DNN) extracts features from input images is still unclear, but it is widely believed that the extraction is performed hierarchically through a process of coarse graining. It reminds us of the basic renormalization group (RG) concept in statistical physics. In order to explore possible relations between DNN and RG, we use the restricted Boltzmann machine (RBM) applied to an Ising model and construct a flow of model parameters (in particular, temperature) generated by the RBM. We show that the unsupervised RBM trained by spin configurations at various temperatures from T=0 to T=6 generates a flow along which the temperature approaches the critical value T_{c}=2.27. This behavior is the opposite of the typical RG flow of the Ising model. By analyzing various properties of the weight matrices of the trained RBM, we discuss why it flows towards T_{c} and how the RBM learns to extract features of spin configurations.
从理论上理解深度神经网络 (DNN) 如何从输入图像中提取特征仍然不清楚，但人们普遍认为，提取是通过粗粒化的过程分层进行的。这让我们想起了统计物理学中基本重整化群 (RG) 的概念。为了探索 DNN 和 RG 之间可能存在的关系，我们使用受限玻尔兹曼机 (RBM) 应用于伊辛模型，并构建了由 RBM 生成的模型参数（特别是温度）的流。我们表明，通过在 T=0 到 T=6 之间的各种温度下的自旋配置来训练无监督的 RBM 会产生沿着温度接近临界值 T_{c}=2.27 的流。这种行为与伊辛模型的典型 RG 流相反。通过分析训练后的 RBM 的权重矩阵的各种性质，我们讨论了为什么它会流向 T_{c}以及 RBM 如何学习提取自旋配置的特征。
