Department of Electrical Engineering, City University of Hong Kong, Tat Chee Avenue, Kowloon, Hong Kong, HKSAR, China; State Key Laboratory of Terahertz and Millimeter Waves, City University of Hong Kong, Tat Chee Avenue, Kowloon, Hong Kong, HKSAR, China; Shenzhen Key Laboratory of Millimeter Wave and Wideband Wireless Communications, CityU Shenzhen Research Institute, Shenzhen, 518057, China.
Department of Electrical Engineering, City University of Hong Kong, Tat Chee Avenue, Kowloon, Hong Kong, HKSAR, China.
Neural Netw. 2024 May;173:106202. doi: 10.1016/j.neunet.2024.106202. Epub 2024 Feb 21.
The concept of randomized neural networks (RNNs), such as the random vector functional link network (RVFL) and extreme learning machine (ELM), is a widely accepted and efficient network method for constructing single-hidden layer feedforward networks (SLFNs). Due to its exceptional approximation capabilities, RNN is being extensively used in various fields. While the RNN concept has shown great promise, its performance can be unpredictable in imperfect conditions, such as weight noises and outliers. Thus, there is a need to develop more reliable and robust RNN algorithms. To address this issue, this paper proposes a new objective function that addresses the combined effect of weight noise and training data outliers for RVFL networks. Based on the half-quadratic optimization method, we then propose a novel algorithm, named noise-aware RNN (NARNN), to optimize the proposed objective function. The convergence of the NARNN is also theoretically validated. We also discuss the way to use the NARNN for ensemble deep RVFL (edRVFL) networks. Finally, we present an extension of the NARNN to concurrently address weight noise, stuck-at-fault, and outliers. The experimental results demonstrate that the proposed algorithm outperforms a number of state-of-the-art robust RNN algorithms.
随机神经网络（RNN）的概念，如随机向量功能链接网络（RVFL）和极限学习机（ELM），是构建单隐层前馈网络（SLFN）的一种广泛接受和有效的网络方法。由于其出色的逼近能力，RNN 正在被广泛应用于各个领域。尽管 RNN 概念表现出了巨大的潜力，但在不完善的条件下，如权重噪声和异常值，其性能可能是不可预测的。因此，需要开发更可靠和稳健的 RNN 算法。为了解决这个问题，本文提出了一个新的目标函数，用于解决 RVFL 网络中权重噪声和训练数据异常值的综合影响。然后，我们基于半二次优化方法，提出了一种新的算法，称为噪声感知 RNN（NARNN），以优化所提出的目标函数。NARNN 的收敛性也在理论上进行了验证。我们还讨论了如何将 NARNN 用于集成深度 RVFL（edRVFL）网络。最后，我们将 NARNN 扩展到同时处理权重噪声、固定故障和异常值。实验结果表明，所提出的算法优于许多现有的鲁棒 RNN 算法。
