Multi-View 3D Object Detection Network for Autonomous Driving

2016/11/23 by Xiaozhi Chen, Chen, Xiaozhi, Huimin Ma +7 · 63 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Autonomous Vehicle Technology and Safety #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Robotics and Sensor-Based Localization #cs.CV

paper · pdf · doi:10.48550/arxiv.1611.07759

To appear in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017

openalex publication_date 2016/11/23 · arxiv created 2017/06/22 · arxiv updated 2017/06/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper aims at high-accuracy 3D object detection in autonomous driving scenario. We propose Multi-View 3D networks (MV3D), a sensory-fusion framework that takes both LIDAR point cloud and RGB images as input and predicts oriented 3D bounding boxes. We encode the sparse 3D point cloud with a compact multi-view representation. The network is composed of two subnetworks: one for 3D object proposal generation and another for multi-view feature fusion. The proposal network generates 3D candidate boxes efficiently from the bird's eye view representation of 3D point cloud. We design a deep fusion scheme to combine region-wise features from multiple views and enable interactions between intermediate layers of different paths. Experiments on the challenging KITTI benchmark show that our approach outperforms the state-of-the-art by around 25% and 30% AP on the tasks of 3D localization and 3D detection. In addition, for 2D detection, our approach obtains 10.3% higher AP than the state-of-the-art on the hard data among the LIDAR-based methods.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• FractalNet: Ultra-Deep Neural Networks without Residuals
• FusionNet: 3D Object Classification Using Multiple Data Representations
• 3D Object Proposals using Stereo Imagery for Accurate Object Class Detection
• 3D Fully Convolutional Network for Vehicle Detection in Point Cloud

Cited by

• LitePT: Lighter Yet Stronger Point Transformer
• Cross-Level Sensor Fusion with Object Lists via Transformer for 3D Object Detection
• Adaptive Dual-Weighted Gravitational Point Cloud Denoising Method
• Active Visual Perception: Opportunities and Challenges
• VG3T: Visual Geometry Grounded Gaussian Transformer
• DriveFlow: Rectified Flow Adaptation for Robust 3D Object Detection in Autonomous Driving
• Rethinking the Encoding and Annotating of 3D Bounding Box: Corner-Aware 3D Object Detection from Point Clouds
• Invisible Triggers, Visible Threats! Road-Style Adversarial Creation Attack for Visual 3D Detection in Autonomous Driving
• How Many Tokens Do 3D Point Cloud Transformer Architectures Really Need?
• M³Detection: Multi-Frame Multi-Level Feature Fusion for Multi-Modal 3D Object Detection with Camera and 4D Imaging Radar
• High-throughput Verticillium wilt detection in cotton: A comparative study of faster R-CNN and YOLOv11
• Adversarially Robust Quantum Transfer Learning
• ALICE-LRI: A General Method for Lossless Range Image Generation for Spinning LiDAR Sensors without Calibration Metadata
• SFGFusion: Surface Fitting Guided 3D Object Detection with 4D Radar and Camera Fusion
• Adversarial Graph Fusion for Incomplete Multi-view Semi-supervised Learning with Tensorial Imputation
• BEVUDA++: Geometric-aware Unsupervised Domain Adaptation for Multi-View 3D Object Detection
• StripDet: Strip Attention-Based Lightweight 3D Object Detection from Point Cloud
• SliceSemOcc: Vertical Slice Based Multimodal 3D Semantic Occupancy Representation
• SAMFusion: Sensor-Adaptive Multimodal Fusion for 3D Object Detection in Adverse Weather
• InfraDiffusion: zero-shot depth map restoration with diffusion models and prompted segmentation from sparse infrastructure point clouds
• BasketLiDAR: The First LiDAR-Camera Multimodal Dataset for Professional Basketball MOT
• You Only Pose Once: A Minimalist's Detection Transformer for Monocular RGB Category-level 9D Multi-Object Pose Estimation
• CMF-IoU: Multi-Stage Cross-Modal Fusion 3D Object Detection with IoU Joint Prediction
• ME³-BEV: Mamba-Enhanced Deep Reinforcement Learning for End-to-End Autonomous Driving with BEV-Perception
• Robust Single-Stage Fully Sparse 3D Object Detection via Detachable Latent Diffusion
• Adaptive LiDAR Scanning: Harnessing Temporal Cues for Efficient 3D Object Detection via Multi-Modal Fusion
• Rethinking Backbone Design for Lightweight 3D Object Detection in LiDAR
• Co-Win: Joint Object Detection and Instance Segmentation in LiDAR Point Clouds via Collaborative Window Processing
• Multistream Network for LiDAR and Camera-based 3D Object Detection in Outdoor Scenes
• Querying Autonomous Vehicle Point Clouds: Enhanced by 3D Object Counting with CounterNet
• Generating Adversarial Point Clouds Using Diffusion Model
• LDRFusion: A LiDAR-Dominant multimodal refinement framework for 3D object detection
• From Binary to Semantic: Utilizing Large-Scale Binary Occupancy Data for 3D Semantic Occupancy Prediction
• Three-Dimensional Millimeter-Wave Imaging Using Active Incoherent Fourier Processing and Pulse Compression
• Evaluation of an Uncertainty-Aware Late Fusion Algorithm for Multi-Source Bird's Eye View Detections Under Controlled Noise
• Rethink 3D Object Detection from Physical World
• Integrating Multi-Modal Sensors: A Review of Fusion Techniques for Intelligent Vehicles
• CSDN: A Context-Gated Self-Adaptive Detection Network for Real-Time Object Detection
• Enhancing point cloud analysis via neighbor aggregation correction based on cross-stage structure correlation
• SASep: Saliency-Aware Structured Separation of Geometry and Feature for Open Set Learning on Point Clouds
• Vision-based Lifting of 2D Object Detections for Automated Driving
• QuadricFormer: Scene as Superquadrics for 3D Semantic Occupancy Prediction
• Fault Localisation and Repair for DL Systems: An Empirical Study with LLMs
• AWML: An Open-Source ML-based Robotics Perception Framework to Deploy for ROS-based Autonomous Driving Software
• 6D Pose Estimation on Point Cloud Data through Prior Knowledge Integration: A Case Study in Autonomous Disassembly
• Point-MoE: Large-Scale Multi-Dataset Training with Mixture-of-Experts for 3D Semantic Segmentation
• Rethinking Gradient-based Adversarial Attacks on Point Cloud Classification
• Fusing Bird View LIDAR Point Cloud and Front View Camera Image for Deep Object Detection
• VLC Fusion: Vision-Language Conditioned Sensor Fusion for Robust Object Detection
• RefPose: Leveraging Reference Geometric Correspondences for Accurate 6D Pose Estimation of Unseen Objects
• Driving Datasets Literature Review
• PointExplainer: Towards Transparent Parkinson's Disease Diagnosis
• Multimodal and Multiview Deep Fusion for Autonomous Marine Navigation
• Learning Multi-view Multi-class Anomaly Detection
• BIMStruct3D: A Fully Automated Hybrid Learning Scan-to-BIM Pipeline with Integrated Topology Refinement
• A Review of 3D Object Detection with Vision-Language Models
• LiDAR-Guided Monocular 3D Object Detection for Long-Range Railway Monitoring
• A Multimodal Hybrid Late-Cascade Fusion Network for Enhanced 3D Object Detection
• Lightweight LiDAR-Camera 3D Dynamic Object Detection and Multi-Class Trajectory Prediction
• Testing the Fault-Tolerance of Multi-Sensor Fusion Perception in Autonomous Driving Systems
• Modeling Local Geometric Structure of 3D Point Clouds using Geo-CNN
• Frustum VoxNet for 3D object detection from RGB-D or Depth images
• TinyCenterSpeed: Efficient Center-Based Object Detection for Autonomous Racing
• Detect Anything 3D in the Wild

Related

• Joint 3D Proposal Generation and Object Detection from View Aggregation
• Multi-View Adaptive Fusion Network for 3D Object Detection
• Stereo RGB and Deeper LIDAR Based Network for 3D Object Detection
• RoIFusion: 3D Object Detection from LiDAR and Vision
• M3D-RPN: Monocular 3D Region Proposal Network for Object Detection