Wang Zifan, Peng Changgen, He Xing, Tan Weijie
State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China.
Guizhou Big Data Academy, Guizhou University, Guiyang 550025, China.
Entropy (Basel). 2023 May 17;25(5):810. doi: 10.3390/e25050810.
Federated learning protects the privacy information in the data set by sharing the average gradient. However, "Deep Leakage from Gradient" (DLG) algorithm as a gradient-based feature reconstruction attack can recover privacy training data using gradients shared in federated learning, resulting in private information leakage. However, the algorithm has the disadvantages of slow model convergence and poor inverse generated images accuracy. To address these issues, a Wasserstein distance-based DLG method is proposed, named WDLG. The WDLG method uses Wasserstein distance as the training loss function achieved to improve the inverse image quality and the model convergence. The hard-to-calculate Wasserstein distance is converted to be calculated iteratively using the Lipschit condition and Kantorovich-Rubinstein duality. Theoretical analysis proves the differentiability and continuity of Wasserstein distance. Finally, experiment results show that the WDLG algorithm is superior to DLG in training speed and inversion image quality. At the same time, we prove through the experiments that differential privacy can be used for disturbance protection, which provides some ideas for the development of a deep learning framework to protect privacy.
联邦学习通过共享平均梯度来保护数据集中的隐私信息。然而，“梯度深度泄漏”（DLG）算法作为一种基于梯度的特征重建攻击，可以利用联邦学习中共享的梯度恢复隐私训练数据，从而导致隐私信息泄露。然而，该算法存在模型收敛速度慢和逆生成图像准确性差的缺点。为了解决这些问题，提出了一种基于瓦瑟斯坦距离的DLG方法，称为WDLG。WDLG方法使用瓦瑟斯坦距离作为训练损失函数，以提高逆图像质量和模型收敛性。通过利普希茨条件和康托罗维奇-鲁宾斯坦对偶性，将难以计算的瓦瑟斯坦距离转换为可迭代计算的形式。理论分析证明了瓦瑟斯坦距离的可微性和连续性。最后，实验结果表明，WDLG算法在训练速度和逆图像质量方面优于DLG。同时，我们通过实验证明了差分隐私可用于干扰保护，这为开发保护隐私的深度学习框架提供了一些思路。
