Mitaim Sanya, Kosko Bart
Department of Electrical Engineering, Faculty of Engineering, Thammasat University, Rangsit Campus, Klong Luang, Pathumthani 12120, Thailand.
IEEE Trans Neural Netw. 2004 Nov;15(6):1526-40. doi: 10.1109/TNN.2004.826218.
Noise can improve how memoryless neurons process signals and maximize their throughput information. Such favorable use of noise is the so-called "stochastic resonance" or SR effect at the level of threshold neurons and continuous neurons. This paper presents theoretical and simulation evidence that 1) lone noisy threshold and continuous neurons exhibit the SR effect in terms of the mutual information between random input and output sequences, 2) a new statistically robust learning law can find this entropy-optimal noise level, and 3) the adaptive SR effect is robust against highly impulsive noise with infinite variance. Histograms estimate the relevant probability density functions at each learning iteration. A theorem shows that almost all noise probability density functions produce some SR effect in threshold neurons even if the noise is impulsive and has infinite variance. The optimal noise level in threshold neurons also behaves nonlinearly as the input signal amplitude increases. Simulations further show that the SR effect persists for several sigmoidal neurons and for Gaussian radial-basis-function neurons.
噪声可以改善无记忆神经元处理信号的方式，并使其吞吐量信息最大化。这种对噪声的有利利用在阈值神经元和连续神经元层面上就是所谓的“随机共振”或SR效应。本文给出了理论和模拟证据，表明：1）单个有噪声的阈值神经元和连续神经元在随机输入与输出序列之间的互信息方面表现出SR效应；2）一种新的具有统计稳健性的学习法则能够找到这个熵最优噪声水平；3）自适应SR效应对于具有无限方差的高度脉冲噪声具有稳健性。在每次学习迭代时，通过直方图估计相关的概率密度函数。一个定理表明，几乎所有噪声概率密度函数在阈值神经元中都会产生某种SR效应，即使噪声是脉冲式的且具有无限方差。随着输入信号幅度增加，阈值神经元中的最优噪声水平也呈现非线性变化。模拟进一步表明，SR效应在多个S型神经元和高斯径向基函数神经元中持续存在。
