Wasserstein GAN for moving differential privacy protection.

Training machine learning models often requires large datasets, but using sensitive data for training poses risks of privacy leakage. Differentially private generative models can synthesize simulated data to prevent privacy breaches. Generative Adversarial Networks (GANs) are widely used for data generation tasks, and GANs with differential privacy can produce data that resembles the distribution of the original sensitive dataset while preventing privacy leaks. However, this often compromises data utility. Balancing data utility with reasonable incorporation of differential privacy is a key challenge in this research area. Traditional differentially private stochastic gradient descent (DP-SGD) algorithms use fixed gradient clipping and noise addition, leading to unstable updates and poor gradient convergence. At present, the advanced privacy protection method of GAN type is GS-WGAN. However, because it is more suitable for decentralized scenarios, distributed training nodes need to communicate frequently with each other, which undoubtedly increases the training cost. This paper proposes a dynamic differential private stochastic gradient descent algorithm (Moving DP-SGD), which combines momentum gradient adjustment with Wasserstein GANs (WGAN). While ensuring differential privacy, it does not adopt cumbersome decentralized scenarios. By using the gradient clipping threshold of progressive synchronous attenuation and the amplitude of noise addition, the training cost was reduced as much as possible, and more usable data was generated. Our method solves the problems of the traditional DP-SGD and demonstrates the efficient and stable generation of differential private data on various image datasets. Compared with the previous methods of adding differential privacy in GAN, our method has achieved outstanding performance in generating privacy-protected and practical-oriented data.