SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training

2021/06/02 by Enmao Diao, Jie Ding, Diao, Enmao +3 · 37 citations
Computer Science · #Artificial intelligence #Bottleneck #Computation #Computer network #Computer science #Cryptography and Data Security #FOS: Computer and information sciences #Federated learning #Ground truth #Labeled data #Machine Learning (cs.LG) #Machine learning #Mobile Crowdsensing and Crowdsourcing #Privacy-Preserving Technologies in Data #Resource (disambiguation) #Task (project management) #Training (meteorology) #Training set #cs.LG

paper · pdf · doi:10.48550/arxiv.2106.01432

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2021/06/02 · arxiv created 2022/10/11 · arxiv updated 2022/10/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

Federated Learning allows the training of machine learning models by using the computation and private data resources of many distributed clients. Most existing results on Federated Learning (FL) assume the clients have ground-truth labels. However, in many practical scenarios, clients may be unable to label task-specific data due to a lack of expertise or resource. We propose SemiFL to address the problem of combining communication-efficient FL such as FedAvg with Semi-Supervised Learning (SSL). In SemiFL, clients have completely unlabeled data and can train multiple local epochs to reduce communication costs, while the server has a small amount of labeled data. We provide a theoretical understanding of the success of data augmentation-based SSL methods to illustrate the bottleneck of a vanilla combination of communication-efficient FL with SSL. To address this issue, we propose alternate training to `fine-tune global model with labeled data' and `generate pseudo-labels with the global model.' We conduct extensive experiments and demonstrate that our approach significantly improves the performance of a labeled server with unlabeled clients training with multiple local epochs. Moreover, our method outperforms many existing SSFL baselines and performs competitively with the state-of-the-art FL and SSL results.

Citations

• Instance Normalization: The Missing Ingredient for Fast Stylization
• Towards Federated Learning at Scale: System Design
• MixMatch: A Holistic Approach to Semi-Supervised Learning
• Temporal Ensembling for Semi-Supervised Learning
• SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum
• FedMD: Heterogenous Federated Learning via Model Distillation
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey
• A Simple Semi-Supervised Learning Framework for Object Detection
• Self-Training With Noisy Student Improves ImageNet Classification
• FedML: A Research Library and Benchmark for Federated Machine Learning
• Learning Multiple Layers of Features from Tiny Images
• Federated Learning Based on Dynamic Regularization

Cited by

• FedSCAl: Leveraging Server and Client Alignment for Unsupervised Federated Source-Free Domain Adaptation
• Semi-Supervised Federated Multi-Label Feature Selection with Fuzzy Information Measures
• SAM-Fed: SAM-Guided Federated Semi-Supervised Learning for Medical Image Segmentation
• CATCHFed: Efficient Unlabeled Data Utilization for Semi-Supervised Federated Learning in Limited Labels Environments
• TACTFL: Temporal Contrastive Training for Multi-modal Federated Learning with Similarity-guided Model Aggregation
• Closer to Reality: Practical Semi-Supervised Federated Learning for Foundation Model Adaptation
• A Semi-Supervised Federated Learning Framework with Hierarchical Clustering Aggregation for Heterogeneous Satellite Networks
• Self-Adaptive and Robust Federated Spectrum Sensing without Benign Majority for Cellular Networks
• Privacy-Preserving Multi-Stage Fall Detection Framework with Semi-supervised Federated Learning and Robotic Vision Confirmation
• Unified Alignment Protocol: Making Sense of the Unlabeled Data in New Domains
• FedEPA: Enhancing Personalization and Modality Alignment in Multimodal Federated Learning

Related

• SemiDFL: A Semi-Supervised Paradigm for Decentralized Federated Learning
• (FL)²: Overcoming Few Labels in Federated Semi-Supervised Learning
• FedSiam: Towards Adaptive Federated Semi-Supervised Learning
• SemiSFL: Split Federated Learning on Unlabeled and Non-IID Data
• FedCon: A Contrastive Framework for Federated Semi-Supervised Learning