Das S, Olurotimi O
Department of Electrical and Computer Engineering, MS 1G5, George Mason University, Fairfax, VA 22030, USA.
IEEE Trans Neural Netw. 1998;9(5):913-36. doi: 10.1109/72.712164.
The classical stochastic analog of the deterministic linear system in engineering is the linear system driven by white noise. This model is the basis of many important engineering methodologies in stochastic control, system identification, and signal estimation, and classification. As the promise of artificial neural networks in modeling nonlinear systems continues to grow, the need for a stochastic analog with quantitative foundations for analysis and synthesis will increase. This paper (along with a companion paper) represent recent work in this direction, examining recurrent neural networks (RNN's) driven by white noise. In this paper, we examine the effect of noise on the typical continuous-time RNN model. First, we perform qualitative analysis establishing uniform boundedness of moments of the neuron states over time. To enable practical application in RNN design and evaluation, however, it is necessary to relate these properties to useful measures that can be estimated. We thus subsequently derive bias and variance measures for the noisy RNN with respect to the corresponding deterministic RNN. This has significant practical implications, since neural-network design is nonminimal in the sense that several different networks can be constructed to solve the same problem. The results in this paper allow the user to quantitatively evaluate given RNN's for noise performance. In addition, the designer can use these results to constrain the design space so that the achieved design satisfies performance specifications whenever possible. An example is provided using the measures derived in this paper to predetermine the best among several RNN designs for a given problem. The companion paper presents results for the discrete-time (so-called time-lagged recurrent) case.
