V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecasting

2023/05/10 by Haibao Yu, Yu, Haibao, Wenxian Yang +23 · 22 citations
Engineering · #Artificial Intelligence (cs.AI) #Autonomous Vehicle Technology and Safety #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Traffic Prediction and Management Techniques #Traffic and Road Safety

paper · pdf · doi:10.48550/arxiv.2305.05938

openalex publication_date 2023/05/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Utilizing infrastructure and vehicle-side information to track and forecast the behaviors of surrounding traffic participants can significantly improve decision-making and safety in autonomous driving. However, the lack of real-world sequential datasets limits research in this area. To address this issue, we introduce V2X-Seq, the first large-scale sequential V2X dataset, which includes data frames, trajectories, vector maps, and traffic lights captured from natural scenery. V2X-Seq comprises two parts: the sequential perception dataset, which includes more than 15,000 frames captured from 95 scenarios, and the trajectory forecasting dataset, which contains about 80,000 infrastructure-view scenarios, 80,000 vehicle-view scenarios, and 50,000 cooperative-view scenarios captured from 28 intersections' areas, covering 672 hours of data. Based on V2X-Seq, we introduce three new tasks for vehicle-infrastructure cooperative (VIC) autonomous driving: VIC3D Tracking, Online-VIC Forecasting, and Offline-VIC Forecasting. We also provide benchmarks for the introduced tasks. Find data, code, and more up-to-date information at \hrefhttps://github.com/AIR-THU/DAIR-V2X-Seqhttps://github.com/AIR-THU/DAIR-V2X-Seq.

Cited by

• SimBEV2X: A Large-Scale Dataset and Data Generation Tool for Multi-Task Vehicle-to-Everything Cooperative Perception
• End-to-End 3-D Spatiotemporal Perception with Multimodal Fusion and V2X Collaboration
• MINT-V2X: A Mobility-Integrated Network Trajectory Dataset for Predictive Resource Management
• CarlaNCAP: A Framework for Quantifying the Safety of Vulnerable Road Users in Infrastructure-Assisted Collective Perception Using EuroNCAP Scenarios
• Advancing Autonomous Driving System Testing: Demands, Challenges, and Future Directions
• SparseCoop: Cooperative Perception with Kinematic-Grounded Queries
• Cooperative Safety Intelligence in V2X-Enabled Transportation: A Survey
• IDEAL-M3D: Instance Diversity-Enriched Active Learning for Monocular 3D Detection
• V2X-RECT: An Efficient V2X Trajectory Prediction Framework via Redundant Interaction Filtering and Tracking Error Correction
• SRA-CP: Spontaneous Risk-Aware Selective Cooperative Perception
• Graph Neural Networks for Vehicular Social Networks: Trends, Challenges, and Opportunities
• V2VLoc: Robust GNSS-Free Collaborative Perception via LiDAR Localization
• UrbanIng-V2X: A Large-Scale Multi-Vehicle, Multi-Infrastructure Dataset Across Multiple Intersections for Cooperative Perception
• CAMNet: Leveraging Cooperative Awareness Messages for Vehicle Trajectory Prediction
• CoPAD : Multi-source Trajectory Fusion and Cooperative Trajectory Prediction with Anchor-oriented Decoder in V2X Scenarios
• Great GATsBi: Hybrid, Multimodal, Trajectory Forecasting for Bicycles using Anticipation Mechanism
• TurboTrain: Towards Efficient and Balanced Multi-Task Learning for Multi-Agent Perception and Prediction
• I2V-GS: Infrastructure-to-Vehicle View Transformation with Gaussian Splatting for Autonomous Driving Data Generation
• MamV2XCalib: V2X-based Target-less Infrastructure Camera Calibration with State Space Model
• Research Challenges and Progress in the End-to-End V2X Cooperative Autonomous Driving Competition
• CoopTrack: Exploring End-to-End Learning for Efficient Cooperative Sequential Perception
• CRUISE: Cooperative Reconstruction and Editing in V2X Scenarios using Gaussian Splatting

Related

• Learning Cooperative Trajectory Representations for Motion Forecasting
• End-to-End Autonomous Driving through V2X Cooperation
• V2X-VLM: End-to-End V2X Cooperative Autonomous Driving Through Large Vision-Language Models
• Unified End-to-End V2X Cooperative Autonomous Driving
• V2X-LLM: Enhancing V2X Integration and Understanding in Connected Vehicle Corridors