Aggregation Design for Personalized Federated Multi-Modal Learning over Wireless Networks

2024/06/12 by Yin, Benshun, Chen, Zhiyong, Tao, Meixia · 1 citation
#FOS: Computer and information sciences #FOS: Electrical engineering #Information Theory (cs.IT) #Signal Processing (eess.SP) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2406.07915

Abstract

Federated Multi-Modal Learning (FMML) is an emerging field that integrates information from different modalities in federated learning to improve the learning performance. In this letter, we develop a parameter scheduling scheme to improve personalized performance and communication efficiency in personalized FMML, considering the non-independent and nonidentically distributed (non-IID) data along with the modality heterogeneity. Specifically, a learning-based approach is utilized to obtain the aggregation coefficients for parameters of different modalities on distinct devices. Based on the aggregation coefficients and channel state, a subset of parameters is scheduled to be uploaded to a server for each modality. Experimental results show that the proposed algorithm can effectively improve the personalized performance of FMML.

Cited by

• Latency-aware Multimodal Federated Learning over UAV Networks

Related

• Analysis and Optimization of Wireless Multimodal Federated Learning on Modal Heterogeneity
• Scheduling and Aggregation Design for Asynchronous Federated Learning over Wireless Networks
• Efficient Wireless Federated Learning with Partial Model Aggregation
• User Scheduling for Federated Learning Through Over-the-Air Computation
• User-Centric Federated Learning