Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting

2024/05/23 by Qingxiang Liu, Xu Liu, Liu, Qingxiang +7 · 14 citations
Computer Science · Decision Sciences · Engineering · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Stock Market Forecasting Methods #Time Series Analysis and Forecasting #Traffic Prediction and Management Techniques

paper · pdf · doi:10.48550/arxiv.2405.14252

openalex publication_date 2024/05/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Unlike natural language processing and computer vision, the development of Foundation Models (FMs) for time series forecasting is blocked due to data scarcity. While recent efforts are focused on building such FMs by unlocking the potential of language models (LMs) for time series analysis, dedicated parameters for various downstream forecasting tasks need training, which hinders the common knowledge sharing across domains. Moreover, data owners may hesitate to share the access to local data due to privacy concerns and copyright protection, which makes it impossible to simply construct a FM on cross-domain training instances. To address these issues, we propose Time-FFM, a Federated Foundation Model for Time series forecasting by leveraging pretrained LMs. Specifically, we begin by transforming time series into the modality of text tokens. To bootstrap LMs for time series reasoning, we propose a prompt adaption module to determine domain-customized prompts dynamically instead of artificially. Given the data heterogeneity across domains, we design a personalized federated training strategy by learning global encoders and local prediction heads. Our comprehensive experiments indicate that Time-FFM outperforms state-of-the-arts and promises effective few-shot and zero-shot forecaster.

Cited by

• Toward Reasoning-Centric Time-Series Analysis
• ARROW: An Adaptive Rollout and Routing Method for Global Weather Forecasting
• Invariant Representation Learning for Source-Free Time Series Forecasting with LLM-Centric Proxy Denoising
• EntroPE: Entropy-Guided Dynamic Patch Encoder for Time Series Forecasting
• AXIS: Explainable Time Series Anomaly Detection with Large Language Models
• CC-Time: Cross-Model and Cross-Modality Time Series Forecasting
• KANMixer: Can KAN Serve as a New Modeling Core for Long-term Time Series Forecasting?
• KFS: KAN based adaptive Frequency Selection learning architecture for long term time series forecasting
• Prioritizing Alignment Paradigms over Task-Specific Model Customization in Time-Series LLMs
• Time Series Forecasting as Reasoning: A Slow-Thinking Approach with Reinforced LLMs
• Enhancing LLM Reasoning for Time Series Classification by Tailored Thinking and Fused Decision
• Towards Cross-Modality Modeling for Time Series Analytics: A Survey in the LLM Era
• Large Language models for Time Series Analysis: Techniques, Applications, and Challenges
• Personalized Federated Sparse Adaptation of Time-Series Foundation Models

Related

• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
• Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting
• A decoder-only foundation model for time-series forecasting
• LLM as Forecasting Planner: Training-Free Text Conditioning for Time-Series Foundation Models
• LLM-PS: Empowering Large Language Models for Time Series Forecasting with Temporal Patterns and Semantics