Communication-Efficient Byzantine-Resilient Federated Zero-Order Optimization

2024/06/20 by Neto, Afonso de Sá Delgado, Egger, Maximilian, Bakshi, Mayank +1 · 1 citation
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2406.14362

Abstract

We introduce CYBER-0, the first zero-order optimization algorithm for memory-and-communication efficient Federated Learning, resilient to Byzantine faults. We show through extensive numerical experiments on the MNIST dataset and finetuning RoBERTa-Large that CYBER-0 outperforms state-of-the-art algorithms in terms of communication and memory efficiency while reaching similar accuracy. We provide theoretical guarantees on its convergence for convex loss functions.

Cited by

• Private Aggregation for Byzantine-Resilient Heterogeneous Federated Learning

Related

• Byzantine-Resilient Zero-Order Optimization for Communication-Efficient Heterogeneous Federated Learning
• Byzantine-Resilient Federated Learning at Edge
• Communication and Energy Efficient Federated Learning using Zero-Order Optimization Technique
• Byzantine-Resilient Federated Learning via Distributed Optimization
• Nesterov-Accelerated Robust Federated Learning Over Byzantine Adversaries