Hillar Christopher J, Tran Ngoc M
Redwood Center for Theoretical Neuroscience, Berkeley, CA, USA.
University of Texas, Austin, Austin, TX, USA.
J Math Neurosci. 2018 Jan 16;8(1):1. doi: 10.1186/s13408-017-0056-2.
The Hopfield recurrent neural network is a classical auto-associative model of memory, in which collections of symmetrically coupled McCulloch-Pitts binary neurons interact to perform emergent computation. Although previous researchers have explored the potential of this network to solve combinatorial optimization problems or store reoccurring activity patterns as attractors of its deterministic dynamics, a basic open problem is to design a family of Hopfield networks with a number of noise-tolerant memories that grows exponentially with neural population size. Here, we discover such networks by minimizing probability flow, a recently proposed objective for estimating parameters in discrete maximum entropy models. By descending the gradient of the convex probability flow, our networks adapt synaptic weights to achieve robust exponential storage, even when presented with vanishingly small numbers of training patterns. In addition to providing a new set of low-density error-correcting codes that achieve Shannon's noisy channel bound, these networks also efficiently solve a variant of the hidden clique problem in computer science, opening new avenues for real-world applications of computational models originating from biology.
霍普菲尔德递归神经网络是一种经典的记忆自联想模型，其中对称耦合的麦卡洛克 - 皮茨二元神经元集合相互作用以执行涌现计算。尽管先前的研究人员已经探索了该网络解决组合优化问题或将重复活动模式作为其确定性动力学吸引子进行存储的潜力，但一个基本的开放性问题是设计一族霍普菲尔德网络，其具有随神经元数量呈指数增长的若干容错记忆。在此，我们通过最小化概率流发现了此类网络，概率流是最近提出的用于估计离散最大熵模型参数的目标。通过沿凸概率流的梯度下降，我们的网络调整突触权重以实现稳健的指数存储，即使在训练模式数量极少的情况下也是如此。除了提供一组新的达到香农噪声信道容量界限的低密度纠错码外，这些网络还能有效解决计算机科学中隐藏团问题的一个变体，为源自生物学的计算模型在现实世界中的应用开辟了新途径。
