DENSE: Data-Free One-Shot Federated Learning

2021/12/23 by Jie Zhang, Chen Chen, Zhang, Jie +12 · 10 citations
Computer Science · #Advanced Graph Neural Networks #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2112.12371

openalex publication_date 2021/12/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

One-shot Federated Learning (FL) has recently emerged as a promising approach, which allows the central server to learn a model in a single communication round. Despite the low communication cost, existing one-shot FL methods are mostly impractical or face inherent limitations, \eg a public dataset is required, clients' models are homogeneous, and additional data/model information need to be uploaded. To overcome these issues, we propose a novel two-stage Data-freE oNe-Shot federated lEarning (DENSE) framework, which trains the global model by a data generation stage and a model distillation stage. DENSE is a practical one-shot FL method that can be applied in reality due to the following advantages: (1) DENSE requires no additional information compared with other methods (except the model parameters) to be transferred between clients and the server; (2) DENSE does not require any auxiliary dataset for training; (3) DENSE considers model heterogeneity in FL, \ie different clients can have different model architectures. Experiments on a variety of real-world datasets demonstrate the superiority of our method.For example, DENSE outperforms the best baseline method Fed-ADI by 5.08% on CIFAR10 dataset.

Cited by

• Rethinking Knowledge Distillation in Collaborative Machine Learning: Memory, Knowledge, and Their Interactions
• TOFA: Training-Free One-Shot Federated Adaptation for Vision-Language Models
• Sparse Model Inversion: Efficient Inversion of Vision Transformers for Data-Free Applications
• Robust Federated Inference
• Global Prompt Refinement with Non-Interfering Attention Masking for One-Shot Federated Learning
• Foundational Models and Federated Learning: Survey, Taxonomy, Challenges and Practical Insights
• Towards a Larger Model via One-Shot Federated Learning on Heterogeneous Client Models
• FedMP: Tackling Medical Feature Heterogeneity in Federated Learning from a Manifold Perspective
• Hypernetworks for Model-Heterogeneous Personalized Federated Learning
• A New One-Shot Federated Learning Framework for Medical Imaging Classification with Feature-Guided Rectified Flow and Knowledge Distillation

Related

• A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning
• One-shot Federated Learning via Synthetic Distiller-Distillate Communication
• Revisiting Ensembling in One-Shot Federated Learning
• Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environments
• Towards One-shot Federated Learning: Advances, Challenges, and Future Directions