Legenstein Robert, Maass Wolfgang
Institute for Theoretical Computer Science, Technische Universitaet Graz, A-8010 Graz, Austria.
Neural Comput. 2008 Jan;20(1):288-309. doi: 10.1162/neco.2008.20.1.288.
The perceptron (also referred to as McCulloch-Pitts neuron, or linear threshold gate) is commonly used as a simplified model for the discrimination and learning capability of a biological neuron. Criteria that tell us when a perceptron can implement (or learn to implement) all possible dichotomies over a given set of input patterns are well known, but only for the idealized case, where one assumes that the sign of a synaptic weight can be switched during learning. We present in this letter an analysis of the classification capability of the biologically more realistic model of a sign-constrained perceptron, where the signs of synaptic weights remain fixed during learning (which is the case for most types of biological synapses). In particular, the VC-dimension of sign-constrained perceptrons is determined, and a necessary and sufficient criterion is provided that tells us when all 2(m) dichotomies over a given set of m patterns can be learned by a sign-constrained perceptron. We also show that uniformity of L(1) norms of input patterns is a sufficient condition for full representation power in the case where all weights are required to be nonnegative. Finally, we exhibit cases where the sign constraint of a perceptron drastically reduces its classification capability. Our theoretical analysis is complemented by computer simulations, which demonstrate in particular that sparse input patterns improve the classification capability of sign-constrained perceptrons.
感知器（也称为麦卡洛克 - 皮茨神经元或线性阈值门）通常被用作生物神经元的辨别和学习能力的简化模型。关于感知器何时能够在给定的一组输入模式上实现（或学习实现）所有可能的二分法的标准是众所周知的，但仅适用于理想化情况，即假设在学习过程中突触权重的符号可以切换。在这封信中，我们对符号约束感知器这种在生物学上更现实的模型的分类能力进行了分析，其中突触权重的符号在学习过程中保持固定（大多数类型的生物突触都是这种情况）。具体而言，确定了符号约束感知器的VC维，并提供了一个充要标准，该标准告诉我们何时一个符号约束感知器能够学习给定的m个模式上的所有2^m种二分法。我们还表明，在所有权重都要求为非负的情况下，输入模式的L1范数的均匀性是具有完全表示能力的充分条件。最后，我们展示了感知器的符号约束会大幅降低其分类能力的情况。我们的理论分析得到了计算机模拟的补充，这些模拟特别表明稀疏输入模式提高了符号约束感知器的分类能力。
