Cui Yuwei, Ahmad Subutai, Hawkins Jeff
Numenta, Inc., Redwood City, CA, United States.
Front Comput Neurosci. 2017 Nov 29;11:111. doi: 10.3389/fncom.2017.00111. eCollection 2017.
Hierarchical temporal memory (HTM) provides a theoretical framework that models several key computational principles of the neocortex. In this paper, we analyze an important component of HTM, the HTM spatial pooler (SP). The SP models how neurons learn feedforward connections and form efficient representations of the input. It converts arbitrary binary input patterns into sparse distributed representations (SDRs) using a combination of competitive Hebbian learning rules and homeostatic excitability control. We describe a number of key properties of the SP, including fast adaptation to changing input statistics, improved noise robustness through learning, efficient use of cells, and robustness to cell death. In order to quantify these properties we develop a set of metrics that can be directly computed from the SP outputs. We show how the properties are met using these metrics and targeted artificial simulations. We then demonstrate the value of the SP in a complete end-to-end real-world HTM system. We discuss the relationship with neuroscience and previous studies of sparse coding. The HTM spatial pooler represents a neurally inspired algorithm for learning sparse representations from noisy data streams in an online fashion.
分层时间记忆（HTM）提供了一个理论框架，该框架对新皮层的几个关键计算原理进行建模。在本文中，我们分析了HTM的一个重要组成部分——HTM空间池化器（SP）。SP模拟了神经元如何学习前馈连接并形成输入的有效表示。它使用竞争性赫布学习规则和稳态兴奋性控制的组合，将任意二进制输入模式转换为稀疏分布式表示（SDR）。我们描述了SP的一些关键特性，包括对不断变化的输入统计数据的快速适应、通过学习提高噪声鲁棒性、细胞的有效利用以及对细胞死亡的鲁棒性。为了量化这些特性，我们开发了一组可以直接从SP输出计算的指标。我们展示了如何使用这些指标和有针对性的人工模拟来满足这些特性。然后，我们在一个完整的端到端真实世界HTM系统中展示了SP的价值。我们讨论了与神经科学的关系以及之前对稀疏编码的研究。HTM空间池化器代表了一种受神经启发的算法，用于以在线方式从噪声数据流中学习稀疏表示。
