Intrinsic Training Signals for Federated Learning Aggregation

2025/07/09 by Cosimo Fiorini, Matteo Mosconi, Fiorini, Cosimo +7
Computer Science · #Artificial Intelligence (cs.AI) #Big Data and Digital Economy #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2507.06813

openalex publication_date 2025/07/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Federated Learning (FL) enables collaborative model training across distributed clients while preserving data privacy. While existing approaches for aggregating client-specific classification heads and adapted backbone parameters require architectural modifications or loss function changes, our method uniquely leverages intrinsic training signals already available during standard optimization. We present LIVAR (Layer Importance and VARiance-based merging), which introduces: i) a variance-weighted classifier aggregation scheme using naturally emergent feature statistics, and ii) an explainability-driven LoRA merging technique based on SHAP analysis of existing update parameter patterns. Without any architectural overhead, LIVAR achieves state-of-the-art performance on multiple benchmarks while maintaining seamless integration with existing FL methods. This work demonstrates that effective model merging can be achieved solely through existing training signals, establishing a new paradigm for efficient federated model aggregation. The code is available at https://github.com/aimagelab/fed-mammoth.

Citations

• Closed-form merging of parameter-efficient modules for Federated Continual Learning
• Federated Class-Incremental Learning with Hierarchical Generative Prototypes
• Mixture of LoRA Experts
• Improving LoRA in Privacy-preserving Federated Learning
• AdaMerging: Adaptive Model Merging for Multi-Task Learning
• LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition
• TIES-Merging: Resolving Interference When Merging Models
• Federated Learning for Predictive Maintenance and Quality Inspection in Industrial Applications
• SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model
• GradMA: A Gradient-Memory-based Accelerated Federated Learning with Alleviated Catastrophic Forgetting
• Dataless Knowledge Fusion by Merging Weights of Language Models
• On the Effectiveness of Parameter-Efficient Fine-Tuning
• Merging Models with Fisher-Weighted Averaging
• LoRA: Low-Rank Adaptation of Large Language Models
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data
• FedProto: Federated Prototype Learning across Heterogeneous Clients
• ImageNet-21K Pretraining for the Masses
• Model-Contrastive Federated Learning
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
• Federated Optimization in Heterogeneous Networks
• EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification
• A Unified Approach to Interpreting Model Predictions
• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Related

• Overcoming label shift with target-aware federated learning
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• FedADP: Unified Model Aggregation for Federated Learning with Heterogeneous Model Architectures
• FedDUAL: A Dual-Strategy with Adaptive Loss and Dynamic Aggregation for Mitigating Data Heterogeneity in Federated Learning
• Federated Learning Aggregation: New Robust Algorithms with Guarantees