Rodriguez P
Department of Cognitive Science, University of California at San Diego, La Jolla, CA 92093, USA.
Neural Comput. 2001 Sep;13(9):2093-118. doi: 10.1162/089976601750399326.
It has been shown that if a recurrent neural network (RNN) learns to process a regular language, one can extract a finite-state machine (FSM) by treating regions of phase-space as FSM states. However, it has also been shown that one can construct an RNN to implement Turing machines by using RNN dynamics as counters. But how does a network learn languages that require counting? Rodriguez, Wiles, and Elman (1999) showed that a simple recurrent network (SRN) can learn to process a simple context-free language (CFL) by counting up and down. This article extends that to show a range of language tasks in which an SRN develops solutions that not only count but also copy and store counting information. In one case, the network stores information like an explicit storage mechanism. In other cases, the network stores information more indirectly in trajectories that are sensitive to slight displacements that depend on context. In this sense, an SRN can learn analog computation as a set of interdependent counters. This demonstrates how SRNs may be an alternative psychological model of language or sequence processing.
研究表明，如果递归神经网络（RNN）学习处理一种正则语言，那么通过将相空间区域视为有限状态机（FSM）状态，就可以提取出一个有限状态机。然而，也有研究表明，可以通过将RNN动态用作计数器来构建一个RNN以实现图灵机。但是，网络如何学习需要计数的语言呢？罗德里格斯、怀尔斯和埃尔曼（1999年）表明，一个简单递归网络（SRN）可以通过向上和向下计数来学习处理一种简单的上下文无关语言（CFL）。本文对此进行了扩展，展示了一系列语言任务，在这些任务中，一个SRN开发出的解决方案不仅能够计数，还能复制和存储计数信息。在一种情况下，网络像显式存储机制一样存储信息。在其他情况下，网络将信息更间接地存储在对依赖于上下文的微小位移敏感的轨迹中。从这个意义上说，一个SRN可以将模拟计算学习为一组相互依赖的计数器。这证明了SRN如何可能成为语言或序列处理的一种替代心理模型。
