Schmitt Michael
Lehrstuhl Mathematik und Informatik, Fakultät für Mathematik Ruhr-Universität Bochum, D-44780 Bochum, Germany.
Neural Comput. 2002 Apr;14(4):919-56. doi: 10.1162/089976602317319018.
Local receptive field neurons comprise such well-known and widely used unit types as radial basis function (RBF) neurons and neurons with center-surround receptive field. We study the Vapnik-Chervonenkis (VC) dimension of feedforward neural networks with one hidden layer of these units. For several variants of local receptive field neurons, we show that the VC dimension of these networks is superlinear. In particular, we establish the bound Omega(W log k) for any reasonably sized network with W parameters and k hidden nodes. This bound is shown to hold for discrete center-surround receptive field neurons, which are physiologically relevant models of cells in the mammalian visual system, for neurons computing a difference of gaussians, which are popular in computational vision, and for standard RBF neurons, a major alternative to sigmoidal neurons in artificial neural networks. The result for RBF neural networks is of particular interest since it answers a question that has been open for several years. The results also give rise to lower bounds for networks with fixed input dimension. Regarding constants, all bounds are larger than those known thus far for similar architectures with sigmoidal neurons. The superlinear lower bounds contrast with linear upper bounds for single local receptive field neurons also derived here.
局部感受野神经元包括径向基函数（RBF）神经元和具有中心 - 环绕感受野的神经元等知名且广泛使用的单元类型。我们研究具有一层由这些单元构成的隐藏层的前馈神经网络的Vapnik - Chervonenkis（VC）维数。对于局部感受野神经元的几种变体，我们表明这些网络的VC维数是超线性的。特别地，对于任何具有W个参数和k个隐藏节点的合理规模网络，我们建立了Ω(W log k)的界。这个界对于离散的中心 - 环绕感受野神经元（哺乳动物视觉系统中细胞的生理相关模型）、计算高斯差分的神经元（在计算视觉中很流行）以及标准RBF神经元（人工神经网络中Sigmoid神经元的主要替代方案）都成立。RBF神经网络的结果特别令人感兴趣，因为它回答了一个已经悬而未决数年的问题。这些结果还给出了具有固定输入维数的网络的下界。关于常数，所有界都比目前已知的具有Sigmoid神经元的类似架构的界更大。这里推导的单个局部感受野神经元的线性上界与超线性下界形成对比。
