Applied Brain Research, Waterloo, ON N2L 3G1, Canada
Cheriton School of Computer Science, University of Waterloo, Waterloo, Ontario, N2L 3G1, Canada
Neural Comput. 2021 Jul 26;33(8):2033-2067. doi: 10.1162/neco_a_01410.
While neural networks are highly effective at learning task-relevant representations from data, they typically do not learn representations with the kind of symbolic structure that is hypothesized to support high-level cognitive processes, nor do they naturally model such structures within problem domains that are continuous in space and time. To fill these gaps, this work exploits a method for defining vector representations that bind discrete (symbol-like) entities to points in continuous topological spaces in order to simulate and predict the behavior of a range of dynamical systems. These vector representations are spatial semantic pointers (SSPs), and we demonstrate that they can (1) be used to model dynamical systems involving multiple objects represented in a symbol-like manner and (2) be integrated with deep neural networks to predict the future of physical trajectories. These results help unify what have traditionally appeared to be disparate approaches in machine learning.
虽然神经网络在从数据中学习与任务相关的表示方面非常有效，但它们通常不会学习具有假设支持高级认知过程的那种符号结构的表示形式，也不会在空间和时间上连续的问题域中自然地对这些结构进行建模。为了弥补这些差距，这项工作利用了一种定义向量表示的方法，该方法将离散（符号样）实体绑定到连续拓扑空间中的点，以模拟和预测一系列动力系统的行为。这些向量表示是空间语义指针（SSP），我们证明它们可以 (1) 用于建模涉及以符号方式表示的多个对象的动力系统，以及 (2) 与深度神经网络集成以预测物理轨迹的未来。这些结果有助于统一机器学习中传统上看起来截然不同的方法。
