TUMTraf V2X Cooperative Perception Dataset

2024/03/02 by Walter Zimmer, Zimmer, Walter, Gerhard Arya Wardana +9 · 28 citations
Computer Science · Engineering · Neuroscience · Psychology · #Brain Tumor Detection and Classification #Computer science #Image Processing and 3D Reconstruction #Medical Imaging and Analysis #Neuroscience #Perception #Psychology

paper · pdf · doi:10.48550/arxiv.2403.01316

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2024/03/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Cooperative perception offers several benefits for enhancing the capabilities of autonomous vehicles and improving road safety. Using roadside sensors in addition to onboard sensors increases reliability and extends the sensor range. External sensors offer higher situational awareness for automated vehicles and prevent occlusions. We propose CoopDet3D, a cooperative multi-modal fusion model, and TUMTraf-V2X, a perception dataset, for the cooperative 3D object detection and tracking task. Our dataset contains 2,000 labeled point clouds and 5,000 labeled images from five roadside and four onboard sensors. It includes 30k 3D boxes with track IDs and precise GPS and IMU data. We labeled eight categories and covered occlusion scenarios with challenging driving maneuvers, like traffic violations, near-miss events, overtaking, and U-turns. Through multiple experiments, we show that our CoopDet3D camera-LiDAR fusion model achieves an increase of +14.36 3D mAP compared to a vehicle camera-LiDAR fusion model. Finally, we make our dataset, model, labeling tool, and dev-kit publicly available on our website: https://tum-traffic-dataset.github.io/tumtraf-v2x.

Cited by

• SimBEV2X: A Large-Scale Dataset and Data Generation Tool for Multi-Task Vehicle-to-Everything Cooperative Perception
• CarlaNCAP: A Framework for Quantifying the Safety of Vulnerable Road Users in Infrastructure-Assisted Collective Perception Using EuroNCAP Scenarios
• Cooperative Safety Intelligence in V2X-Enabled Transportation: A Survey
• SRA-CP: Spontaneous Risk-Aware Selective Cooperative Perception
• ELiC: Efficient LiDAR Geometry Compression via Cross-Bit-depth Feature Propagation and Bag-of-Encoders
• MIC-BEV: Multi-Infrastructure Camera Bird's-Eye-View Transformer with Relation-Aware Fusion for 3D Object Detection
• UrbanIng-V2X: A Large-Scale Multi-Vehicle, Multi-Infrastructure Dataset Across Multiple Intersections for Cooperative Perception
• DRCP: Diffusion on Reinforced Cooperative Perception for Perceiving Beyond Limits
• CoPAD : Multi-source Trajectory Fusion and Cooperative Trajectory Prediction with Anchor-oriented Decoder in V2X Scenarios
• Safety-Critical Learning for Long-Tail Events: The TUM Traffic Accident Dataset
• SpaRC-AD: A Baseline for Radar-Camera Fusion in End-to-End Autonomous Driving
• TurboTrain: Towards Efficient and Balanced Multi-Task Learning for Multi-Agent Perception and Prediction
• MamV2XCalib: V2X-based Target-less Infrastructure Camera Calibration with State Space Model
• Research Challenges and Progress in the End-to-End V2X Cooperative Autonomous Driving Competition
• CoopTrack: Exploring End-to-End Learning for Efficient Cooperative Sequential Perception
• HeCoFuse: Cross-Modal Complementary V2X Cooperative Perception with Heterogeneous Sensors
• TruckV2X: A Truck-Centered Perception Dataset
• Multimodal HD Mapping for Intersections by Intelligent Roadside Units
• 2.5D Object Detection for Intelligent Roadside Infrastructure
• Evaluation of an Uncertainty-Aware Late Fusion Algorithm for Multi-Source Bird's Eye View Detections Under Controlled Noise
• A Late Collaborative Perception Framework for 3D Multi-Object and Multi-Source Association and Fusion
• Integrating Multi-Modal Sensors: A Review of Fusion Techniques for Intelligent Vehicles
• AirV2X: Unified Air-Ground Vehicle-to-Everything Collaboration
• A Synthetic Benchmark for Collaborative 3D Semantic Occupancy Prediction in V2X-Enabled Autonomous Driving
• Real-World Deployment of Cloud-based Autonomous Mobility Systems for Outdoor and Indoor Environments
• M3CAD: Towards Generic Cooperative Autonomous Driving Benchmark
• Decentralized Fusion of 3D Extended Object Tracking based on a B-Spline Shape Model
• S2S-Net: Addressing the Domain Gap of Heterogeneous Sensor Systems in LiDAR-Based Collective Perception
• Systematic Literature Review on Vehicular Collaborative Perception -- A Computer Vision Perspective
• Collaborative Perception Datasets for Autonomous Driving: A Review

Related

• Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Clouds
• V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception
• V2X-Real: a Large-Scale Dataset for Vehicle-to-Everything Cooperative Perception
• InScope: A New Real-world 3D Infrastructure-side Collaborative Perception Dataset for Open Traffic Scenarios
• DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection