Personalized Federated Sparse Adaptation of Time-Series Foundation Models

2026/08/05 by Priyanka Nihalchandani, Naman Srivastava, Varun Ojha +1
Computer Science · Mathematics · #cs.LG #cs.AI #stat.ML

paper · pdf

15 pages

arxiv created 2026/08/05 · arxiv updated 2026/08/06

Abstract

Federated adaptation of time-series foundation models (TSFMs) is attractive for building energy forecasting because meter data are private, distributed, and highly non-IID. However, a single parameter-sharing strategy is unlikely to serve all pretrained TSFMs or building clients: fully shared adapters can suppress building-specific temporal behavior, while fully local adaptation discards cross-building transfer. We propose a personalized federated sparse adaptation framework with a heterogeneous temporal mixture-of-experts (MoE) adapter placed after the pretrained TSFM representation. A sequence-level router maps each 168-hour context window to a top-k subset of experts specialized for periodicity, long-range interactions, local variation, trend-residual structure, and multi-resolution behavior. We compare global FL, local training, and personalized FL variants with globally shared or client-private expert banks. Across 50 buildings and three TSFM backbones, personalization consistently outperforms Global FL-MoE and Local MoE, while the best sparse-adaptation strategy varies by backbone and metric. Routing behavior further reveals client-level expert specialization, expert concentration, and near-uniform routing across backbones, showing that federated TSFM adaptation should be both client-aware and backbone-aware.

Citations

• Federated Foundation Models on Heterogeneous Time Series
• TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecasting
• Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts
• Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting
• MOMENT: A Family of Open Time-series Foundation Models
• Unified Training of Universal Time Series Forecasting Transformers
• FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting
• Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
• LoRA: Low-Rank Adaptation of Large Language Models
• FedBABU: Towards Enhanced Representation for Federated Image Classification
• Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
• GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
• The Building Data Genome Project 2, energy meter data from the ASHRAE Great Energy Predictor III competition
• Personalized Federated Learning with Moreau Envelopes
• Federated Learning with Personalization Layers
• Federated Optimization in Heterogeneous Networks
• Attention Is All You Need
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning