Adaptive complex-valued stepsize based fast learning of complex-valued neural networks.

Complex-valued gradient descent algorithm is a popular tool to optimize functions of complex variables, especially for the training of complex-valued neural networks. However, the choice of suitable learning stepsize is a challenging task during the training process. In this paper, an adaptive complex-valued stepsize design method is proposed for complex-valued neural networks by generalizing the adaptable learning rate tree technique to the complex domain. The scaling and rotation factors are introduced to simultaneously adjust the amplitude and phase of complex-valued stepsize. The search range is thus expanded from half line to half plane such that better search direction is obtained at each iteration. We analyze the dynamics of the algorithm near a saddle point and find that it is very easy to escape from the saddle point to guarantee fast convergence and high accuracy. Some experimental results on function approximation and pattern classification tasks are presented to illustrate the advantages of the proposed algorithm over some previous ones.