HAVT-IVD: Heterogeneity-Aware Cross-Modal Network for Audio-Visual Surveillance: Idling Vehicles Detection With Multichannel Audio and Multiscale Visual Cues

2025/04/15 by Xiwen Li, Li, Xiwen, Tolga Taşdizen +2
Computer Science · Engineering · #Autonomous Vehicle Technology and Safety #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Image Enhancement Techniques #Robotics (cs.RO) #Video Surveillance and Tracking Methods

paper · pdf · doi:10.48550/arxiv.2504.16102

openalex publication_date 2025/04/15 · openalex created_date 2025/10/11 · openalex updated_date 2026/07/30

Abstract

Idling vehicle detection (IVD) uses surveillance video and multichannel audio to localize and classify vehicles in the last frame as moving, idling, or engine-off in pick-up zones. IVD faces three challenges: (i) modality heterogeneity between visual cues and audio patterns; (ii) large box scale variation requiring multi-resolution detection; and (iii) training instability due to coupled detection heads. The previous end-to-end (E2E) model with simple CBAM-based bi-modal attention fails to handle these issues and often misses vehicles. We propose HAVT-IVD, a heterogeneity-aware network with a visual feature pyramid and decoupled heads. Experiments show HAVT-IVD improves mAP by 7.66 over the disjoint baseline and 9.42 over the E2E baseline.

Related

• Joint Audio-Visual Idling Vehicle Detection with Streamlined Input Dependencies
• Visual Dominance and Emerging Multimodal Approaches in Distracted Driving Detection: A Review of Machine Learning Techniques
• Real-Time Driver Monitoring Systems through Modality and View Analysis
• IC3M: In-Car Multimodal Multi-object Monitoring for Abnormal Status of Both Driver and Passengers
• MEVDT: Multi-Modal Event-Based Vehicle Detection and Tracking Dataset