Kansai University, Takatsuki, Osaka, Japan.
Neural Netw. 2010 Sep;23(7):926-38. doi: 10.1016/j.neunet.2010.04.008. Epub 2010 May 6.
The neocognitron, which was proposed by Fukushima (1980), is a hierarchical multi-layered neural network capable of robust visual pattern recognition. It acquires the ability to recognize patterns through learning. This paper proposes a new rule for competitive learning, named winner-kill-loser, and apply it to the neocognitron. The winner-kill-loser rule resembles the winner-take-all rule. Every time when a training stimulus is presented, non-silent cells compete with each other. The winner, however, not only takes all, but also kills losers. In other words, the winner learns the training stimulus, and losers are removed from the network. If all cells are silent, a new cell is generated and it learns the training stimulus. Thus feature-extracting cells gradually come to distribute uniformly in the feature space. The use of winner-kill-loser rule is not limited to the neocognitron. It is useful for various types of competitive learning, in general. This paper also proposes several improvements made on the neocognitron: such as, disinhibition to the inhibitory surround in the connections to C-cells (or complex cells) from S-cells (or simple cells); and square root shaped saturation in the input-to-output characteristics of C-cells. As a result of these improvements, the recognition rate of the neocognitron has been largely increased.
新认知机是由福岛（1980 年）提出的一种分层多层神经网络，能够进行稳健的视觉模式识别。它通过学习获得识别模式的能力。本文提出了一种新的竞争学习规则，称为胜者-杀-败者，并将其应用于新认知机。胜者-杀-败者规则类似于胜者全拿规则。每次呈现训练刺激时，非静默细胞相互竞争。然而，胜者不仅全拿，而且还会杀死败者。换句话说，胜者学习训练刺激，而败者则从网络中删除。如果所有细胞都沉默，就会生成一个新细胞并学习训练刺激。因此，特征提取细胞逐渐在特征空间中均匀分布。胜者-杀-败者规则的使用不仅限于新认知机。它通常对各种类型的竞争学习都很有用。本文还对新认知机提出了几项改进：例如，在 S 细胞（或简单细胞）到 C 细胞（或复杂细胞）的连接中抑制抑制性环绕；以及 C 细胞输入-输出特性的平方根形饱和。由于这些改进，新认知机的识别率大大提高。
