Private Federated Learning Without a Trusted Server: Optimal Algorithms\n for Convex Losses

2021/06/17 by Andrew M. Lowy, Meisam Razaviyayn, Lowy, Andrew +1 · 4 citations
Computer Science · Medicine · #Cryptography and Data Security #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Patient Dignity and Privacy #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2106.09779

openalex publication_date 2021/06/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper studies federated learning (FL)--especially cross-silo FL--with\ndata from people who do not trust the server or other silos. In this setting,\neach silo (e.g. hospital) has data from different people (e.g. patients) and\nmust maintain the privacy of each person's data (e.g. medical record), even if\nthe server or other silos act as adversarial eavesdroppers. This requirement\nmotivates the study of Inter-Silo Record-Level Differential Privacy (ISRL-DP),\nwhich requires silos' communications to satisfy record/item-level differential\nprivacy (DP). ISRL-DP ensures that the data of each person (e.g. patient) in\nsilo i (e.g. hospital i) cannot be leaked. ISRL-DP is different from\nwell-studied privacy notions. Central and user-level DP assume that people\ntrust the server/other silos. On the other end of the spectrum, local DP\nassumes that people do not trust anyone at all (even their own silo). Sitting\nbetween central and local DP, ISRL-DP makes the realistic assumption (in\ncross-silo FL) that people trust their own silo, but not the server or other\nsilos. In this work, we provide tight (up to logarithms) upper and lower bounds\nfor ISRL-DP FL with convex/strongly convex loss functions and homogeneous\n(i.i.d.) silo data. Remarkably, we show that similar bounds are attainable for\nsmooth losses with arbitrary heterogeneous silo data distributions, via an\naccelerated ISRL-DP algorithm. We also provide tight upper and lower bounds for\nISRL-DP federated empirical risk minimization, and use acceleration to attain\nthe optimal bounds in fewer rounds of communication than the state-of-the-art.\nFinally, with a secure "shuffler" to anonymize silo messages (but without a\ntrusted server), our algorithm attains the optimal central DP rates under more\npractical trust assumptions. Numerical experiments show favorable\nprivacy-accuracy tradeoffs for our algorithm in classification and regression\ntasks.\n

Citations

• Smooth minimization of non-smooth functions
• Differentially Private Federated Learning: A Client Level Perspective
• Federated Learning With Differential Privacy: Algorithms and Performance Analysis
• Differentially Private ADMM for Convex Distributed Learning: Improved Accuracy via Multi-Step Approximation
• Analyzing User-Level Privacy Attack Against Federated Learning

Cited by

• On the Gradient Complexity of Private Optimization with Private Oracles
• Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practice
• Federated Learning of Quantile Inference under Local Differential Privacy
• On the Inherent Privacy of Zeroth Order Projected Gradient Descent

Related

• Private Heterogeneous Federated Learning Without a Trusted Server Revisited: Error-Optimal and Communication-Efficient Algorithms for Convex Losses
• Private Non-Convex Federated Learning Without a Trusted Server
• A Stochastic Optimization Framework for Private and Fair Learning From Decentralized Data
• Private and Federated Stochastic Convex Optimization: Efficient Strategies for Centralized Systems
• Local Differential Privacy for Federated Learning