2025/07/23 by Lingdong Kong, Dongyue Lu, Kong, Lingdong +15 · 4 citations
Decision Sciences · #Computer Vision and Pattern Recognition (cs.CV) #Data Quality and Management #FOS: Computer and information sciences #Robotics (cs.RO) #Scientific Computing and Data Management
paper · pdf · doi:10.48550/arxiv.2507.17664
openalex publication_date 2025/07/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Event cameras offer microsecond-level latency and robustness to motion blur, making them ideal for understanding dynamic environments. Yet, connecting these asynchronous streams to human language remains an open challenge. We introduce Talk2Event, the first large-scale benchmark for language-driven object grounding in event-based perception. Built from real-world driving data, we provide over 30,000 validated referring expressions, each enriched with four grounding attributes -- appearance, status, relation to viewer, and relation to other objects -- bridging spatial, temporal, and relational reasoning. To fully exploit these cues, we propose EventRefer, an attribute-aware grounding framework that dynamically fuses multi-attribute representations through a Mixture of Event-Attribute Experts (MoEE). Our method adapts to different modalities and scene dynamics, achieving consistent gains over state-of-the-art baselines in event-only, frame-only, and event-frame fusion settings. We hope our dataset and approach will establish a foundation for advancing multimodal, temporally-aware, and language-driven perception in real-world robotics and autonomy.