Li Cen-Jhih, Huang Pin-Han, Ma Yi-Ting, Hung Hung, Huang Su-Yun
Institute of Statistical Science, Academia Sinica, Taipei City 11529, Taiwan.
Data Science Degree Program, National Taiwan University, Taipei City 10617, Taiwan.
Entropy (Basel). 2022 May 13;24(5):686. doi: 10.3390/e24050686.
Federated learning is a framework for multiple devices or institutions, called local clients, to collaboratively train a global model without sharing their data. For federated learning with a central server, an aggregation algorithm integrates model information sent from local clients to update the parameters for a global model. Sample mean is the simplest and most commonly used aggregation method. However, it is not robust for data with outliers or under the Byzantine problem, where Byzantine clients send malicious messages to interfere with the learning process. Some robust aggregation methods were introduced in literature including marginal median, geometric median and trimmed-mean. In this article, we propose an alternative robust aggregation method, named γ-mean, which is the minimum divergence estimation based on a robust density power divergence. This γ-mean aggregation mitigates the influence of Byzantine clients by assigning fewer weights. This weighting scheme is data-driven and controlled by the γ value. Robustness from the viewpoint of the influence function is discussed and some numerical results are presented.
联邦学习是一种适用于多个设备或机构（称为本地客户端）的框架，用于在不共享数据的情况下协作训练全局模型。对于具有中央服务器的联邦学习，一种聚合算法会整合从本地客户端发送的模型信息，以更新全局模型的参数。样本均值是最简单且最常用的聚合方法。然而，对于存在异常值的数据或在拜占庭问题（即拜占庭客户端发送恶意消息以干扰学习过程）的情况下，它并不稳健。文献中介绍了一些稳健的聚合方法，包括边际中位数、几何中位数和截尾均值。在本文中，我们提出了一种替代的稳健聚合方法，称为γ均值，它是基于稳健密度幂散度的最小散度估计。这种γ均值聚合通过分配较少的权重来减轻拜占庭客户端的影响。这种加权方案是数据驱动的，并由γ值控制。从影响函数的角度讨论了稳健性，并给出了一些数值结果。
