Institut für Mathematische Stochastik, Fachbereich 10: Mathematik und Informatik, Westfälische Wilhelms-Universität Münster, Orléans-Ring 10, 48149 Münster, Germany.
Neural Netw. 2022 Apr;148:121-128. doi: 10.1016/j.neunet.2022.01.006. Epub 2022 Jan 21.
The realization function of a shallow ReLU network is a continuous and piecewise affine function f:R→R, where the domain R is partitioned by a set of n hyperplanes into cells on which f is affine. We show that the minimal representation for f uses either n, n+1 or n+2 neurons and we characterize each of the three cases. In the particular case, where the input layer is one-dimensional, minimal representations always use at most n+1 neurons but in all higher dimensional settings there are functions for which n+2 neurons are needed. Then we show that the set of minimal networks representing f forms a C-submanifold M and we derive the dimension and the number of connected components of M. Additionally, we give a criterion for the hyperplanes that guarantees that a continuous, piecewise affine function is the realization function of an appropriate shallow ReLU network.
浅层 ReLU 网络的实现函数是一个连续的分段仿射函数 f:R→R，其中 R 的定义域被一组 n 个超平面划分为 f 是仿射的单元。我们证明了 f 的最小表示使用 n、n+1 或 n+2 个神经元，并对三种情况进行了特征描述。在输入层是一维的特殊情况下，最小表示始终使用最多 n+1 个神经元，但在所有更高维的情况下，都有需要 n+2 个神经元的函数。然后我们证明了表示 f 的最小网络的集合形成一个 C-子流形 M，并推导出 M 的维数和连通分量的数量。此外，我们给出了一个保证连续分段仿射函数是适当的浅层 ReLU 网络的实现函数的超平面的准则。
