Oh Sangchul, Baggag Abdelkader, Nha Hyunchul
Qatar Environment and Energy Research Institute, Hamad Bin Khalifa University, Qatar Foundation, 5825 Doha, Qatar.
Qatar Computing Research Institute, Hamad Bin Khalifa University, Qatar Foundation, 5825 Doha, Qatar.
Entropy (Basel). 2020 May 11;22(5):538. doi: 10.3390/e22050538.
A restricted Boltzmann machine is a generative probabilistic graphic network. A probability of finding the network in a certain configuration is given by the Boltzmann distribution. Given training data, its learning is done by optimizing the parameters of the energy function of the network. In this paper, we analyze the training process of the restricted Boltzmann machine in the context of statistical physics. As an illustration, for small size bar-and-stripe patterns, we calculate thermodynamic quantities such as entropy, free energy, and internal energy as a function of the training epoch. We demonstrate the growth of the correlation between the visible and hidden layers via the subadditivity of entropies as the training proceeds. Using the Monte-Carlo simulation of trajectories of the visible and hidden vectors in the configuration space, we also calculate the distribution of the work done on the restricted Boltzmann machine by switching the parameters of the energy function. We discuss the Jarzynski equality which connects the path average of the exponential function of the work and the difference in free energies before and after training.
受限玻尔兹曼机是一种生成式概率图形网络。在特定配置下找到该网络的概率由玻尔兹曼分布给出。给定训练数据，其学习过程是通过优化网络能量函数的参数来完成的。在本文中，我们在统计物理学的背景下分析受限玻尔兹曼机的训练过程。作为示例，对于小尺寸的条形和条纹图案，我们计算诸如熵、自由能和内能等热力学量作为训练轮次的函数。我们通过随着训练进行熵的次可加性来证明可见层和隐藏层之间相关性的增长。使用配置空间中可见向量和隐藏向量轨迹的蒙特卡罗模拟，我们还通过切换能量函数的参数来计算在受限玻尔兹曼机上所做工作的分布。我们讨论了连接工作的指数函数的路径平均值与训练前后自由能差的雅津斯基等式。
