He Chuan, Peng Le, Sun Ju
Department of Computer Science and Engineering, University of Minnesota.
Transact Mach Learn Res. 2024;2024. Epub 2024 May 3.
In practice, many machine learning (ML) problems come with constraints, and their applied domains involve distributed sensitive data that cannot be shared with others, e.g., in healthcare. Collaborative learning in such practical scenarios entails federated learning (FL) for ML problems with constraints, or for short. Despite the extensive developments of FL techniques in recent years, these techniques only deal with unconstrained FL problems or FL problems with simple constraints that are amenable to easy projections. There is little work dealing with FL problems with general constraints. To fill this gap, we take the first step toward building an algorithmic framework for solving FL problems with general constraints. In particular, we propose a new FL algorithm for constrained ML problems based on the proximal augmented Lagrangian (AL) method. Assuming convex objective and convex constraints plus other mild conditions, we establish the worst-case complexity of the proposed algorithm. Our numerical experiments show the effectiveness of our algorithm in performing Neyman-Pearson classification and fairness-aware learning with nonconvex constraints, in an FL setting.
在实际应用中，许多机器学习（ML）问题都存在约束条件，并且其应用领域涉及无法与他人共享的分布式敏感数据，例如在医疗保健领域。在这种实际场景下的协作学习需要针对有约束的ML问题进行联邦学习（FL），简而言之就是这样。尽管近年来FL技术有了广泛的发展，但这些技术仅处理无约束的FL问题或具有易于投影的简单约束的FL问题。处理具有一般约束的FL问题的工作很少。为了填补这一空白，我们朝着构建一个用于解决具有一般约束的FL问题的算法框架迈出了第一步。具体而言，我们基于近端增广拉格朗日（AL）方法，提出了一种用于约束ML问题的新FL算法。假设目标函数为凸函数且约束条件为凸约束以及其他一些温和条件，我们建立了所提算法的最坏情况复杂度。我们的数值实验表明，在FL设置下，我们的算法在执行具有非凸约束的奈曼 - 皮尔逊分类和公平感知学习方面是有效的。
