Over-the-Air Federated Multi-Task Learning

2021/06/27 by Ma, Haoming, Yuan, Xiaojun, Fan, Dian +3
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Networking and Internet Architecture (cs.NI)

paper · doi:10.48550/arxiv.2106.14229

Abstract

In this letter, we introduce over-the-air computation into the communication design of federated multi-task learning (FMTL), and propose an over-the-air federated multi-task learning (OA-FMTL) framework, where multiple learning tasks deployed on edge devices share a non-orthogonal fading channel under the coordination of an edge server (ES). Specifically, the model updates for all the tasks are transmitted and superimposed concurrently over a non-orthogonal uplink fading channel, and the model aggregations of all the tasks are reconstructed at the ES through a modified version of the turbo compressed sensing algorithm (Turbo-CS) that overcomes inter-task interference. Both convergence analysis and numerical results show that the OA-FMTL framework can significantly improve the system efficiency in terms of reducing the number of channel uses without causing substantial learning performance degradation.

Related

• Over-the-Air Federated Multi-Task Learning via Model Sparsification and Turbo Compressed Sensing
• Over-the-Air Computation Empowered Federated Learning: A Joint Uplink-Downlink Design
• Over-the-Air Federated Learning: Rethinking Edge AI Through Signal Processing
• Multi-cell Non-coherent Over-the-Air Computation for Federated Edge Learning
• Over-the-Air Federated Multi-Task Learning Over MIMO Multiple Access Channels