Shen Xiaoxi, Jiang Chang, Sakhanenko Lyudamila, Lu Qing
Texas State University, San Marcos, TX, USA.
University of Florida, Gainesville, FL, USA.
J Nonparametr Stat. 2023;35(4):839-868. doi: 10.1080/10485252.2023.2209218. Epub 2023 May 13.
Neural networks have become one of the most popularly used methods in machine learning and artificial intelligence. Due to the universal approximation theorem (Hornik et al., 1989), a neural network with one hidden layer can approximate any continuous function on compact support as long as the number of hidden units is sufficiently large. Statistically, a neural network can be classified into a nonlinear regression framework. However, if we consider it parametrically, due to the unidentifiability of the parameters, it is difficult to derive its asymptotic properties. Instead, we consider the estimation problem in a nonparametric regression framework and use the results from sieve estimation to establish the consistency, the rates of convergence and the asymptotic normality of the neural network estimators. We also illustrate the validity of the theories via simulations.
神经网络已成为机器学习和人工智能中最常用的方法之一。根据泛逼近定理（霍尼克等人，1989年），只要隐藏单元的数量足够大，具有一个隐藏层的神经网络就可以逼近紧致支撑集上的任何连续函数。从统计学角度来看，神经网络可归类为非线性回归框架。然而，如果从参数角度考虑，由于参数的不可识别性，很难推导出其渐近性质。相反，我们在非参数回归框架中考虑估计问题，并利用筛法估计的结果来建立神经网络估计量的一致性、收敛速度和渐近正态性。我们还通过模拟来说明这些理论的有效性。
