National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210093, China.
Neural Netw. 2022 Jul;151:80-93. doi: 10.1016/j.neunet.2022.03.024. Epub 2022 Mar 29.
Complex-valued neural networks have attracted increasing attention in recent years, while it remains open on the advantages of complex-valued neural networks in comparison with real-valued networks. This work takes one step on this direction by introducing the complex-reaction network with fully-connected feed-forward architecture. We prove the universal approximation property for complex-reaction networks, and show that a class of radial functions can be approximated by a complex-reaction network using the polynomial number of parameters, whereas real-valued networks need at least exponential parameters to reach the same approximation level. For empirical risk minimization, we study the landscape and convergence of complex gradient descents. Our theoretical result shows that the critical point set of complex-reaction networks is a proper subset of that of real-valued networks, which may show some insights on finding the optimal solutions more easily for complex-reaction networks.
近年来，复值神经网络引起了越来越多的关注，但其相对于实值网络的优势仍有待探讨。本工作通过引入全连接前馈结构的复反应网络，在这一方向上迈出了一步。我们证明了复反应网络的通用逼近性质，并表明一类径向函数可以使用复反应网络的多项式数量的参数来逼近，而实值网络需要至少指数数量的参数才能达到相同的逼近水平。对于经验风险最小化，我们研究了复梯度下降的景观和收敛性。我们的理论结果表明，复反应网络的临界点集是实值网络临界点集的真子集，这可能为更容易找到复反应网络的最优解提供一些启示。
