Sum John, Leung Chi-Sing, Ho Kevin I-J
IEEE Trans Neural Netw Learn Syst. 2018 Sep;29(9):4212-4222. doi: 10.1109/TNNLS.2017.2759905. Epub 2017 Oct 27.
In this paper, the effect of input noise, output node stochastic, and recurrent state noise on the Wang $k$ WTA is analyzed. Here, we assume that noise exists at the recurrent state $y(t)$ and it can either be additive or multiplicative. Besides, its dynamical change (i.e., $dy/dt$ ) is corrupted by noise as well. In sequel, we model the dynamics of $y(t)$ as a stochastic differential equation and show that the stochastic behavior of $y(t)$ is equivalent to an Ito diffusion. Its stationary distribution is a Gibbs distribution, whose modality depends on the noise condition. With moderate input noise and very small recurrent state noise, the distribution is single modal and hence $y(\infty )$ has high probability varying within the input values of the $k$ and $k+1$ winners (i.e., correct output). With small input noise and large recurrent state noise, the distribution could be multimodal and hence $y(\infty )$ could have probability varying outside the input values of the $k$ and $k+1$ winners (i.e., incorrect output). In this regard, we further derive the conditions that the $k$ WTA has high probability giving correct output. Our results reveal that recurrent state noise could have severe effect on Wang $k$ WTA. But, input noise and output node stochastic could alleviate such an effect.
本文分析了输入噪声、输出节点随机性和递归状态噪声对Wang $k$ 胜者全得（WTA）的影响。在此，我们假设在递归状态$y(t)$处存在噪声，它可以是加性的或乘性的。此外，其动态变化（即$dy/dt$）也受到噪声的干扰。接下来，我们将$y(t)$的动态建模为一个随机微分方程，并表明$y(t)$的随机行为等同于伊藤扩散。其平稳分布是一个吉布斯分布，其模态取决于噪声条件。在适度的输入噪声和非常小的递归状态噪声下，分布是单峰的，因此$y(\infty)$很有可能在$k$个和$k + 1$个胜者的输入值范围内变化（即正确输出）。在小输入噪声和大递归状态噪声下，分布可能是多峰的，因此$y(\infty)$可能有概率在$k$个和$k + 1$个胜者的输入值范围之外变化（即错误输出）。在这方面，我们进一步推导了$k$ WTA有高概率给出正确输出的条件。我们的结果表明，递归状态噪声可能对Wang $k$ WTA有严重影响。但是，输入噪声和输出节点随机性可以减轻这种影响。
