Super-Linear: A Lightweight Pretrained Mixture of Linear Experts for Time Series Forecasting

2025/09/18 by Liran Nochumsohn, Nochumsohn, Liran, Raz Marshanski +5 · 1 citation
Computer Science · Decision Sciences · #FOS: Computer and information sciences #Forecasting Techniques and Applications #Machine Learning (cs.LG) #Stock Market Forecasting Methods #Time Series Analysis and Forecasting

paper · pdf · doi:10.48550/arxiv.2509.15105

openalex publication_date 2025/09/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Time series forecasting (TSF) is critical in domains like energy, finance, healthcare, and logistics, requiring models that generalize across diverse datasets. Large pre-trained models such as Chronos and Time-MoE show strong zero-shot (ZS) performance but suffer from high computational costs. In this work, we introduce Super-Linear, a lightweight and scalable mixture-of-experts (MoE) model for general forecasting. It replaces deep architectures with simple frequency-specialized linear experts, trained on resampled data across multiple frequency regimes. A lightweight spectral gating mechanism dynamically selects relevant experts, enabling efficient, accurate forecasting. Despite its simplicity, Super-Linear demonstrates strong performance across benchmarks, while substantially improving efficiency, robustness to sampling rates, and interpretability. The implementation of Super-Linear is available at: \hrefhttps://github.com/azencot-group/SuperLinearhttps://github.com/azencot-group/SuperLinear.

Citations

• TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning
• Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting
• A Multi-Task Learning Approach to Linear Multivariate Forecasting
• Sundial: A Family of Highly Capable Time Series Foundation Models
• Tailored Forecasting from Short Time Series via Meta-learning
• FreqMoE: Enhancing Time Series Forecasting through Frequency Decomposition Mixture of Experts
• DUET: Dual Clustering Enhanced Multivariate Time Series Forecasting
• KEDformer:Knowledge Extraction Seasonal Trend Decomposition for Long-term Sequence Prediction
• GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation
• Timer-XL: Long-Context Transformers for Unified Time Series Forecasting
• CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns
• Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts
• VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters
• UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting
• TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment
• Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer
• SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters
• An Analysis of Linear Time Series Forecasting Models
• Chronos: Learning the Language of Time Series
• MOMENT: A Family of Open Time-series Foundation Models
• Timer: Generative Pre-trained Transformers Are Large Time Series Models
• Unified Training of Universal Time Series Forecasting Transformers
• Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series
• AutoMixer for Improved Multivariate Time-Series Forecasting on Business and IT Observability Data
• UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting
• A decoder-only foundation model for time-series forecasting
• iTransformer: Inverted Transformers Are Effective for Time Series Forecasting
• TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
• FITS: Modeling Time Series with 10k Parameters
• One Fits All:Power General Time Series Analysis by Pretrained LM
• A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT
• A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
• TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
• Are Transformers Effective for Time Series Forecasting?
• CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting
• N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting
• NHITS: Neural Hierarchical Interpolation for Time Series Forecasting
• FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting
• Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
• Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
• Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
• Temporal Fusion Transformers for Interpretable Multi-horizon Time Series\n Forecasting
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Forecasting at Scale
• Attention Is All You Need
• DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks
• Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks
• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
• Adaptive Mixtures of Local Experts
• LibCity: A Unified Library Towards Efficient and Comprehensive Urban Spatial-Temporal Prediction
• Revisiting Long-term Time Series Forecasting: An Investigation on Linear Mapping

Cited by

• Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting

Related

• MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models
• Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts
• GateTS: Versatile and Efficient Forecasting via Attention-Inspired routed Mixture-of-Experts
• LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting
• Ada-MoGE: Adaptive Mixture of Gaussian Expert Model for Time Series Forecasting