Liu Enze, Chu Zhiguang, Zhang Xing
School of Electronics and Information Engineering, Liaoning University of Technology, Shiying Street 169, Jinzhou, 121000, Liaoning, China.
Key Laboratory of Security for Network and Data in Industrial Internet of Liaoning Province, Shiying Street 169, Jinzhou, 121000, Liaoning, China.
Sci Rep. 2025 Jun 4;15(1):19634. doi: 10.1038/s41598-025-03178-2.
Training machine learning models often requires large datasets, but using sensitive data for training poses risks of privacy leakage. Differentially private generative models can synthesize simulated data to prevent privacy breaches. Generative Adversarial Networks (GANs) are widely used for data generation tasks, and GANs with differential privacy can produce data that resembles the distribution of the original sensitive dataset while preventing privacy leaks. However, this often compromises data utility. Balancing data utility with reasonable incorporation of differential privacy is a key challenge in this research area. Traditional differentially private stochastic gradient descent (DP-SGD) algorithms use fixed gradient clipping and noise addition, leading to unstable updates and poor gradient convergence. At present, the advanced privacy protection method of GAN type is GS-WGAN. However, because it is more suitable for decentralized scenarios, distributed training nodes need to communicate frequently with each other, which undoubtedly increases the training cost. This paper proposes a dynamic differential private stochastic gradient descent algorithm (Moving DP-SGD), which combines momentum gradient adjustment with Wasserstein GANs (WGAN). While ensuring differential privacy, it does not adopt cumbersome decentralized scenarios. By using the gradient clipping threshold of progressive synchronous attenuation and the amplitude of noise addition, the training cost was reduced as much as possible, and more usable data was generated. Our method solves the problems of the traditional DP-SGD and demonstrates the efficient and stable generation of differential private data on various image datasets. Compared with the previous methods of adding differential privacy in GAN, our method has achieved outstanding performance in generating privacy-protected and practical-oriented data.
训练机器学习模型通常需要大量数据集，但使用敏感数据进行训练存在隐私泄露风险。差分隐私生成模型可以合成模拟数据以防止隐私泄露。生成对抗网络（GAN）被广泛用于数据生成任务，具有差分隐私的GAN可以生成类似于原始敏感数据集分布的数据，同时防止隐私泄露。然而，这往往会损害数据效用。在合理纳入差分隐私的同时平衡数据效用是该研究领域的一个关键挑战。传统的差分隐私随机梯度下降（DP-SGD）算法使用固定的梯度裁剪和噪声添加，导致更新不稳定且梯度收敛性差。目前，GAN类型的先进隐私保护方法是GS-WGAN。然而，由于它更适合分散场景，分布式训练节点需要相互频繁通信，这无疑增加了训练成本。本文提出了一种动态差分隐私随机梯度下降算法（Moving DP-SGD），它将动量梯度调整与瓦瑟斯坦生成对抗网络（WGAN）相结合。在确保差分隐私的同时，它不采用繁琐的分散场景。通过使用渐进同步衰减的梯度裁剪阈值和噪声添加幅度，尽可能降低了训练成本，并生成了更多可用数据。我们的方法解决了传统DP-SGD的问题，并在各种图像数据集上展示了差分隐私数据的高效稳定生成。与之前在GAN中添加差分隐私的方法相比,我们的方法在生成隐私保护且面向实际的数据方面取得了出色的性能。
