Du Jie, Li Wei, Liu Peng, Vong Chi-Man, You Yongke, Lei Baiying, Wang Tianfu
National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518060, Guangdong, China.
Artificial Intelligence Industrial Innovation Research Center, Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen, 518110, China.
Neural Netw. 2024 Oct;178:106409. doi: 10.1016/j.neunet.2024.106409. Epub 2024 May 24.
Multi-center disease diagnosis aims to build a global model for all involved medical centers. Due to privacy concerns, it is infeasible to collect data from multiple centers for training (i.e., centralized learning). Federated Learning (FL) is a decentralized framework that enables multiple clients (e.g., medical centers) to collaboratively train a global model while retaining patient data locally for privacy. However, in practice, the data across medical centers are not independently and identically distributed (Non-IID), causing two challenging issues: (1) catastrophic forgetting at clients, i.e., the local model at clients will forget the knowledge received from the global model after local training, causing reduced performance; and (2) invalid aggregation at the server, i.e., the global model at the server may not be favorable to some clients after model aggregation, resulting in a slow convergence rate. To mitigate these issues, an innovative Federated learning using Model Projection (FedMoP) is proposed, which guarantees: (1) the loss of local model on global data does not increase after local training without accessing the global data so that the performance will not be degenerated; and (2) the loss of global model on local data does not increase after aggregation without accessing local data so that convergence rate can be improved. Extensive experimental results show that our FedMoP outperforms state-of-the-art FL methods in terms of accuracy, convergence rate and communication cost. In particular, our FedMoP also achieves comparable or even higher accuracy than centralized learning. Thus, our FedMoP can ensure privacy protection while outperforming centralized learning in accuracy and communication cost.
多中心疾病诊断旨在为所有参与的医疗中心构建一个全局模型。由于隐私问题，从多个中心收集数据进行训练（即集中式学习）是不可行的。联邦学习（FL）是一种去中心化框架，它使多个客户端（例如医疗中心）能够协作训练全局模型，同时将患者数据本地保留以保护隐私。然而，在实际中，医疗中心之间的数据并非独立同分布（非IID），这导致了两个具有挑战性的问题：（1）客户端的灾难性遗忘，即客户端的本地模型在本地训练后会忘记从全局模型获得的知识，导致性能下降；（2）服务器端的无效聚合，即服务器端的全局模型在模型聚合后可能对某些客户端不利，导致收敛速度缓慢。为了缓解这些问题，提出了一种创新的使用模型投影的联邦学习（FedMoP），它保证：（1）在不访问全局数据的情况下，本地模型在全局数据上的损失在本地训练后不会增加，从而性能不会退化；（2）在不访问本地数据的情况下，全局模型在本地数据上的损失在聚合后不会增加，从而可以提高收敛速度。大量实验结果表明，我们的FedMoP在准确性、收敛速度和通信成本方面优于现有最先进的联邦学习方法。特别是，我们的FedMoP在准确性方面也达到了与集中式学习相当甚至更高的水平。因此，我们的FedMoP可以确保隐私保护，同时在准确性和通信成本方面优于集中式学习。
