Towards Privacy-Preserving and Heterogeneity-aware Split Federated Learning via Probabilistic Masking

2025/09/18 by Xingchen Wang, Feijie Wu, Wang, Xingchen +13
Computer Science · #Adversarial Robustness in Machine Learning #Cryptography and Data Security #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2509.14603

openalex publication_date 2025/09/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Split Federated Learning (SFL) has emerged as an efficient alternative to traditional Federated Learning (FL) by reducing client-side computation through model partitioning. However, exchanging of intermediate activations and model updates introduces significant privacy risks, especially from data reconstruction attacks that recover original inputs from intermediate representations. Existing defenses using noise injection often degrade model performance. To overcome these challenges, we present PM-SFL, a scalable and privacy-preserving SFL framework that incorporates Probabilistic Mask training to add structured randomness without relying on explicit noise. This mitigates data reconstruction risks while maintaining model utility. To address data heterogeneity, PM-SFL employs personalized mask learning that tailors submodel structures to each client's local data. For system heterogeneity, we introduce a layer-wise knowledge compensation mechanism, enabling clients with varying resources to participate effectively under adaptive model splitting. Theoretical analysis confirms its privacy protection, and experiments on image and wireless sensing tasks demonstrate that PM-SFL consistently improves accuracy, communication efficiency, and robustness to privacy attacks, with particularly strong performance under data and system heterogeneity.

Citations

• FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction
• Towards Federated RLHF with Aggregated Client Preference for LLMs
• FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuning
• Revisiting Gradient Pruning: A Dual Realization for Defending against Gradient Attacks
• SkyMask: Attack-agnostic Robust Federated Learning with Fine-grained Learnable Masks
• FedExP: Speeding Up Federated Averaging via Extrapolation
• Sparse Random Networks for Communication-Efficient Federated Learning
• Concealing Sensitive Samples against Gradient Leakage in Federated Learning
• Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices
• Anchor Sampling for Federated Learning with Partial Client Participation
• Gradient Obfuscation Gives a False Sense of Security in Federated Learning
• Differentially Private Label Protection in Split Learning
• From Deterioration to Acceleration: A Calibration Approach to Rehabilitating Step Asynchronism in Federated Optimization
• Parameterized Knowledge Transfer for Personalized Federated Learning
• Privacy Amplification Via Bernoulli Sampling
• Exploiting Shared Representations for Personalized Federated Learning
• Provable Defense against Privacy Leakage in Federated Learning from Representation Perspective
• Privacy-preserving Collaborative Learning with Automatic Transformation Search
• InstaHide: Instance-hiding Schemes for Private Distributed Learning
• HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients
• NoPeek: Information leakage reduction to share activations in distributed deep learning
• LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasets
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• SplitFed: When Federated Learning Meets Split Learning
• Think Locally, Act Globally: Federated Learning with Local and Global Representations
• Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
• On the Convergence of FedAvg on Non-IID Data
• Deep Leakage from Gradients
• Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask
• LEAF: A Benchmark for Federated Settings
• Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences
• How To Backdoor Federated Learning
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• Certified Robustness to Adversarial Examples with Differential Privacy
• mixup: Beyond Empirical Risk Minimization
• Deep Residual Learning for Image Recognition
• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

Related

• Privacy and Efficiency of Communications in Federated Split Learning
• Federated Learning with Enhanced Privacy via Model Splitting and Random Client Participation
• FedRP: A Communication-Efficient Approach for Differentially Private Federated Learning Using Random Projection
• Enhancing Security Using Random Binary Weights in Privacy-Preserving Federated Learning
• Privacy-Preserving Federated Learning on Partitioned Attributes