School of Mathematics and Statistics, Xidian University, Xi'an 710071, PR China.
School of Aerospace Science and Technology, Xidian University, Xi'an 710071, PR China.
Neural Netw. 2019 Oct;118:300-309. doi: 10.1016/j.neunet.2018.10.014. Epub 2018 Nov 14.
This paper aims to propose a distributed semi-supervised learning (D-SSL) algorithm to solve D-SSL problems, where training samples are often extremely large-scale and located on distributed nodes over communication networks. Training data of each node consists of labeled and unlabeled samples whose output values or labels are unknown. These nodes communicate in a distributed way, where each node has only access to its own data and can only exchange local information with its neighboring nodes. In some scenarios, these distributed data cannot be processed centrally. As a result, D-SSL problems cannot be centrally solved by using traditional semi-supervised learning (SSL) algorithms. The state-of-the-art D-SSL algorithm, denoted as Distributed Laplacian Regularization Least Square (D-LapRLS), is a kernel based algorithm. It is essential for the D-LapRLS algorithm to estimate the global Euclidian Distance Matrix (EDM) with respect to total samples, which is time-consuming especially when the scale of training data is large. In order to solve D-SSL problems and overcome the common drawback of kernel based D-SSL algorithms, we propose a novel Manifold Regularization (MR) based D-SSL algorithm using Wavelet Neural Network (WNN) and Zero-Gradient-Sum (ZGS) distributed optimization strategy. Accordingly, each node is assigned an individual WNN with the same basis functions. In order to initialize the proposed D-SSL algorithm, we propose a centralized MR based SSL algorithm using WNN. We denote the proposed SSL and D-SSL algorithms as Laplacian WNN (LapWNN) and distributed LapWNN (D-LapWNN), respectively. The D-LapWNN algorithm works in a fully distributed fashion by using ZGS strategy, whose convergence is guaranteed by the Lyapunov method. During the learning process, each node only exchanges local coefficients with its neighbors rather than raw data. It means that the D-LapWNN algorithm is a privacy preserving method. At last, several illustrative simulations are presented to show the efficiency and advantage of the proposed algorithm.
本文旨在提出一种分布式半监督学习（D-SSL）算法来解决 D-SSL 问题，其中训练样本通常是大规模的，并且分布在通信网络上的分布式节点上。每个节点的训练数据由有标签和无标签的样本组成，其输出值或标签是未知的。这些节点以分布式的方式进行通信，其中每个节点只能访问其自身的数据，并且只能与其相邻节点交换本地信息。在某些情况下，这些分布式数据不能集中处理。因此，D-SSL 问题不能通过使用传统的半监督学习（SSL）算法来集中解决。最新的 D-SSL 算法，称为分布式拉普拉斯正则化最小二乘法（D-LapRLS），是一种基于核的算法。对于 D-LapRLS 算法来说，估计关于总样本的全局欧几里得距离矩阵（EDM）是至关重要的，这在训练数据规模较大时非常耗时。为了解决 D-SSL 问题并克服基于核的 D-SSL 算法的常见缺点，我们提出了一种基于流形正则化（MR）的新型 D-SSL 算法，该算法使用小波神经网络（WNN）和零梯度和（ZGS）分布式优化策略。相应地，为每个节点分配一个具有相同基函数的单独 WNN。为了初始化所提出的 D-SSL 算法，我们提出了一种基于 WNN 的集中式 MR 基于 SSL 算法。我们将所提出的 SSL 和 D-SSL 算法分别表示为拉普拉斯 WNN（LapWNN）和分布式 LapWNN（D-LapWNN）。D-LapWNN 算法通过使用 ZGS 策略以完全分布式的方式工作，其收敛性由 Lyapunov 方法保证。在学习过程中，每个节点仅与其邻居交换本地系数，而不是原始数据。这意味着 D-LapWNN 算法是一种保护隐私的方法。最后，展示了几个说明性的仿真结果，以显示所提出算法的效率和优势。
