Mix2FLD: Downlink Federated Learning After Uplink Federated Distillation With Two-Way Mixup

2020/06/17 by Seungeun Oh, Oh, Seungeun, Jihong Park +9 · 3 citations
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mobile Crowdsensing and Crowdsourcing #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2006.09801

openalex publication_date 2020/06/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This letter proposes a novel communication-efficient and privacy-preserving distributed machine learning framework, coined Mix2FLD. To address uplink-downlink capacity asymmetry, local model outputs are uploaded to a server in the uplink as in federated distillation (FD), whereas global model parameters are downloaded in the downlink as in federated learning (FL). This requires a model output-to-parameter conversion at the server, after collecting additional data samples from devices. To preserve privacy while not compromising accuracy, linearly mixed-up local samples are uploaded, and inversely mixed up across different devices at the server. Numerical evaluations show that Mix2FLD achieves up to 16.7% higher test accuracy while reducing convergence time by up to 18.8% under asymmetric uplink-downlink channels compared to FL.

Citations

• Distilling On-Device Intelligence at the Network Edge
• Advances and Open Problems in Federated Learning

Cited by

• P²U: Progressive Precision Update For Efficient Model Distribution
• Improving Convergence for Semi-Federated Learning: An Energy-Efficient Approach by Manipulating Over-the-Air Distortion
• Federated Learning-Distillation Alternation for Resource-Constrained IoT

Related

• Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
• DistDD: Distributed Data Distillation Aggregation through Gradient Matching
• Federated Knowledge Distillation
• SplitFed: When Federated Learning Meets Split Learning
• FedH2L: Federated Learning with Model and Statistical Heterogeneity