Tiny-TSM: Efficiently Training a Lightweight SOTA Time Series Foundation Model

2025/11/24 by Birkel, Felix
Computer Science · Decision Sciences · Engineering · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Stock Market Forecasting Methods #Time Series Analysis and Forecasting #Traffic Prediction and Management Techniques

paper · doi:10.48550/arxiv.2511.19272

openalex publication_date 2025/11/24 · openalex created_date 2025/11/27 · openalex updated_date 2026/07/28

Abstract

We present Tiny-TSM, a time series foundation model characterized by small scale, economical training, and state-of-the-art performance. It comprises 23M total parameters, trained on a single A100 GPU in less than a week using a new synthetic data generation and data augmentation pipeline (SynthTS). Without any neural architecture search, hyperparameter tuning, or scaling up model size, Tiny-TSM achieves state-of-the-art performance on a wide range of time series benchmark datasets, often outperforming much larger models and even matching the performance of much larger, industrial-scale, likely highly tuned foundation models. Specifically, Tiny-TSM outperforms all other time series foundation models we evaluated on medium- and long-term forecasting tasks under MSE loss, while short-term accuracy is still competitive with state-of-the-art models. We also introduce a causal input normalization scheme that enables time series models to be trained with dense next-token prediction loss, significantly accelerating convergence speed and reducing training time. All experiments were conducted on a single A100 GPU, illustrating the practicality of the proposed approach in a resource-constrained setting.

Citations

• TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning
• Benchmarking Pre-Trained Time Series Models for Electricity Price Forecasting
• From Tables to Time: Extending TabPFN-v2 to Time Series Forecasting
• GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation
• Toto: Time Series Optimized Transformer for Observability
• Unified Training of Universal Time Series Forecasting Transformers
• A decoder-only foundation model for time-series forecasting
• A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
• TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second
• Masked Autoencoders Are Scalable Vision Learners
• Learning Transferable Visual Models From Natural Language Supervision
• Language Models are Few-Shot Learners
• Deep Learning for Time Series Forecasting: The Electric Load Case
• Traffic Flow Combination Forecasting Method Based on Improved LSTM and ARIMA
• N-BEATS: Neural basis expansion analysis for interpretable time series forecasting
• Recurrent Neural Networks for Multivariate Time Series with Missing Values
• Featureless Classification of Light Curves

Related

• Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series
• SVTime: Small Time Series Forecasting Models Informed by "Physics" of Large Vision Model Forecasters
• Less is More: Unlocking Specialization of Time Series Foundation Models via Structured Pruning
• SEMPO: Lightweight Foundation Models for Time Series Forecasting
• The Few Govern the Many:Unveiling Few-Layer Dominance for Time Series Models