A Pretrained Probabilistic Transformer for City-Scale Traffic Volume Prediction

2025/06/03 by Shiyu Shen, Shen, Shiyu, Bin Pan +3
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Traffic Prediction and Management Techniques #Traffic control and management

paper · pdf · doi:10.48550/arxiv.2506.02654

openalex publication_date 2025/06/03 · openalex created_date 2025/10/14 · openalex updated_date 2026/07/28

Abstract

City-scale traffic volume prediction plays a pivotal role in intelligent transportation systems, yet remains a challenge due to the inherent incompleteness and bias in observational data. Although deep learning-based methods have shown considerable promise, most existing approaches produce deterministic point estimates, thereby neglecting the uncertainty arising from unobserved traffic flows. Furthermore, current models are typically trained in a city-specific manner, which hinders their generalizability and limits scalability across diverse urban contexts. To overcome these limitations, we introduce TrafficPPT, a Pretrained Probabilistic Transformer designed to model traffic volume as a distributional aggregation of trajectories. Our framework fuses heterogeneous data sources-including real-time observations, historical trajectory data, and road network topology-enabling robust and uncertainty-aware traffic inference. TrafficPPT is initially pretrained on large-scale simulated data spanning multiple urban scenarios, and later fine-tuned on target cities to ensure effective domain adaptation. Experiments on real-world datasets show that TrafficPPT consistently surpasses state-of-the-art baselines, particularly under conditions of extreme data sparsity. Code will be open.

Citations

• Explainable Trajectory Representation through Dictionary Learning
• Reconstructing Transit Vehicle Trajectory Using High-Resolution GPS Data
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
• GPT-4 Technical Report
• CBLab: Supporting the Training of Large-scale Traffic Control Policies with Scalable Traffic Simulation
• Masked Autoencoders Are Scalable Vision Learners
• View Vertically: A Hierarchical Network for Trajectory Prediction via Fourier Spectrums
• Spatio-Temporal Hybrid Graph Convolutional Network for Traffic Forecasting in Telecommunication Networks
• GPT-GNN: Generative Pre-Training of Graph Neural Networks
• Learning Probabilistic Intersection Traffic Models for Trajectory Prediction
• Strategies for Pre-training Graph Neural Networks
• OSMnx: New methods for acquiring, constructing, analyzing, and visualizing complex street networks
• Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction
• Long short-term memory neural network for traffic speed prediction using remote microwave sensor data
• On the Distribution of Probe Traffic Volume Estimated without Trajectory Reconstruction
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Related

• OpenCity: Open Spatio-Temporal Foundation Models for Traffic Prediction
• TrTr: A Versatile Pre-Trained Large Traffic Model based on Transformer for Capturing Trajectory Diversity in Vehicle Population
• FlowDistill: Scalable Traffic Flow Prediction via Distillation from LLMs
• Traffic Prediction with Transfer Learning: A Mutual Information-based Approach
• Uncertainty Quantification for Image-based Traffic Prediction across Cities