VALSE 论文速览 第186期：自适应差分隐私的动态个性化联邦学习 ...

论文题目：

Dynamic Personalized Federated Learning with Adaptive Differential Privacy

作者列表：

杨熙元 (武汉大学)、黄文柯 (武汉大学)、叶茫 (武汉大学)

B站观看网址：

https://www.bilibili.com/video/BV1Ws421T7x6/

论文摘要：

Personalized federated learning with differential privacy has been considered a feasible solution to address non-IID distribution of data and privacy leakage risks. However, current personalized federated learning methods suffer from inflexible personalization and convergence difficulties due to two main factors: 1) Firstly, we observe that the prevailing personalization methods mainly achieve this by personalizing a fixed portion of the model, which lacks flexibility. 2) Moreover, we further demonstrate that the default gradient calculation is sensitive to the widely used clipping operations in differential privacy, resulting in difficulties in convergence. Considering that Fisher information values can serve as an effective measure for estimating the information content of parameters by reflecting the model sensitivity to parameters, we aim to leverage this property to address the aforementioned challenges. In this paper, we propose a novel federated learning method with Dynamic Fisher Personalization and Adaptive Constraint (FedDPA) to handle these challenges. Firstly, by using layer-wise Fisher information to measure the information content of local parameters, we retain local parameters with high Fisher values during the personalization process, which are considered informative, simultaneously prevent these parameters from noise perturbation. Secondly, we introduce an adaptive approach by applying differential constraint strategies to personalized parameters and shared parameters identified in the previous for better convergence.  Our method boosts performance through flexible personalization while mitigating the slow convergence caused by clipping operations. Experimental results on CIFAR-10, FEMNIST and SVHN dataset demonstrate the effectiveness of our approach in achieving better performance and robustness against clipping, under personalized federated learning with differential privacy.

参考文献：

[1] Xiyuan Yang, Wenke Huang, Mang Ye. Dynamic Personalized Federated Learning with Adaptive Differential Privacy. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS), 2023.

论文链接：

[https://openreview.net/pdf?id=RteNLuc8D9]

代码链接：

[https://github.com/xiyuanyang45/DynamicPFL]

视频讲者简介：

杨熙元，武汉大学大三本科生，导师是叶茫教授，主要研究方向为联邦学习。目前在CCF-A类会议上发表多篇论文。

个人主页：

https://xiyuanyang45.github.io

特别鸣谢本次论文速览主要组织者：

月度轮值AC：王一帆 (大连理工大学)