Burbank Kendra S
Department of Statistics, University of Chicago, Chicago, Illinois, United States of America.
PLoS Comput Biol. 2015 Dec 3;11(12):e1004566. doi: 10.1371/journal.pcbi.1004566. eCollection 2015 Dec.
The autoencoder algorithm is a simple but powerful unsupervised method for training neural networks. Autoencoder networks can learn sparse distributed codes similar to those seen in cortical sensory areas such as visual area V1, but they can also be stacked to learn increasingly abstract representations. Several computational neuroscience models of sensory areas, including Olshausen & Field's Sparse Coding algorithm, can be seen as autoencoder variants, and autoencoders have seen extensive use in the machine learning community. Despite their power and versatility, autoencoders have been difficult to implement in a biologically realistic fashion. The challenges include their need to calculate differences between two neuronal activities and their requirement for learning rules which lead to identical changes at feedforward and feedback connections. Here, we study a biologically realistic network of integrate-and-fire neurons with anatomical connectivity and synaptic plasticity that closely matches that observed in cortical sensory areas. Our choice of synaptic plasticity rules is inspired by recent experimental and theoretical results suggesting that learning at feedback connections may have a different form from learning at feedforward connections, and our results depend critically on this novel choice of plasticity rules. Specifically, we propose that plasticity rules at feedforward versus feedback connections are temporally opposed versions of spike-timing dependent plasticity (STDP), leading to a symmetric combined rule we call Mirrored STDP (mSTDP). We show that with mSTDP, our network follows a learning rule that approximately minimizes an autoencoder loss function. When trained with whitened natural image patches, the learned synaptic weights resemble the receptive fields seen in V1. Our results use realistic synaptic plasticity rules to show that the powerful autoencoder learning algorithm could be within the reach of real biological networks.
自编码器算法是一种简单但强大的用于训练神经网络的无监督方法。自编码器网络可以学习到类似于在诸如视觉区域V1等皮质感觉区域中所见的稀疏分布式编码，但它们也可以堆叠起来以学习越来越抽象的表示。包括奥尔沙森和菲尔德的稀疏编码算法在内的几种感觉区域的计算神经科学模型，可以被视为自编码器的变体，并且自编码器在机器学习社区中已经得到了广泛应用。尽管自编码器功能强大且用途广泛，但一直难以以生物学上现实的方式实现。挑战包括它们需要计算两个神经元活动之间的差异以及它们对学习规则的要求，这些规则会导致前馈和反馈连接处发生相同的变化。在这里，我们研究了一个具有解剖学连接性和突触可塑性的生物学现实的积分发放神经元网络，该网络与在皮质感觉区域中观察到的情况紧密匹配。我们对突触可塑性规则的选择受到最近实验和理论结果的启发，这些结果表明反馈连接处的学习可能与前馈连接处的学习形式不同，并且我们的结果关键取决于这种可塑性规则的新颖选择。具体而言，我们提出前馈与反馈连接处的可塑性规则是尖峰时间依赖可塑性（STDP）的时间上相反的版本，从而导致一种我们称为镜像STDP（mSTDP）的对称组合规则。我们表明，使用mSTDP，我们的网络遵循一种学习规则，该规则近似地最小化自编码器损失函数。当用白化的自然图像块进行训练时，学习到的突触权重类似于在V1中看到的感受野。我们的结果使用现实的突触可塑性规则表明，强大的自编码器学习算法可能是真实生物网络能够实现的。
