Cheng Song, Chen Jing, Wang Lei
Institute of Physics, Chinese Academy of Sciences, Beijing 100190, China.
School of Physical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China.
Entropy (Basel). 2018 Aug 7;20(8):583. doi: 10.3390/e20080583.
We compare and contrast the statistical physics and quantum physics inspired approaches for unsupervised generative modeling of classical data. The two approaches represent probabilities of observed data using energy-based models and quantum states, respectively. Classical and quantum information patterns of the target datasets therefore provide principled guidelines for structural design and learning in these two approaches. Taking the Restricted Boltzmann Machines (RBM) as an example, we analyze the information theoretical bounds of the two approaches. We also estimate the classical mutual information of the standard MNIST datasets and the quantum Rényi entropy of corresponding Matrix Product States (MPS) representations. Both information measures are much smaller compared to their theoretical upper bound and exhibit similar patterns, which imply a common inductive bias of low information complexity. By comparing the performance of RBM with various architectures on the standard MNIST datasets, we found that the RBM with local sparse connection exhibit high learning efficiency, which supports the application of tensor network states in machine learning problems.
我们比较并对比了受统计物理学和量子物理学启发的用于经典数据无监督生成建模的方法。这两种方法分别使用基于能量的模型和量子态来表示观测数据的概率。因此，目标数据集的经典和量子信息模式为这两种方法的结构设计和学习提供了原则性指导。以受限玻尔兹曼机（RBM）为例，我们分析了这两种方法的信息理论界限。我们还估计了标准MNIST数据集的经典互信息以及相应矩阵乘积态（MPS）表示的量子雷尼熵。与它们的理论上限相比，这两种信息度量都要小得多，并且呈现出相似的模式，这意味着存在低信息复杂度的共同归纳偏差。通过比较不同架构的RBM在标准MNIST数据集上的性能，我们发现具有局部稀疏连接的RBM表现出较高的学习效率，这支持了张量网络态在机器学习问题中的应用。
