Temporal-contextual Event Learning for Pedestrian Crossing Intent Prediction

2025/04/04 by H. Liang, Hezhe Qiao, Liang, Hongbin +9
Computer Science · Engineering · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Autonomous Vehicle Technology and Safety #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Traffic and Road Safety

paper · pdf · doi:10.48550/arxiv.2504.06292

openalex publication_date 2025/04/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Ensuring the safety of vulnerable road users through accurate prediction of pedestrian crossing intention (PCI) plays a crucial role in the context of autonomous and assisted driving. Analyzing the set of observation video frames in ego-view has been widely used in most PCI prediction methods to forecast the cross intent. However, they struggle to capture the critical events related to pedestrian behaviour along the temporal dimension due to the high redundancy of the video frames, which results in the sub-optimal performance of PCI prediction. Our research addresses the challenge by introducing a novel approach called \underlineTemporal-\underlinecontextual Event \underlineLearning (TCL). The TCL is composed of the Temporal Merging Module (TMM), which aims to manage the redundancy by clustering the observed video frames into multiple key temporal events. Then, the Contextual Attention Block (CAB) is employed to adaptively aggregate multiple event features along with visual and non-visual data. By synthesizing the temporal feature extraction and contextual attention on the key information across the critical events, TCL can learn expressive representation for the PCI prediction. Extensive experiments are carried out on three widely adopted datasets, including PIE, JAAD-beh, and JAAD-all. The results show that TCL substantially surpasses the state-of-the-art methods. Our code can be accessed at https://github.com/dadaguailhb/TCL.

Citations

• VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
• Masked Autoencoders Are Scalable Vision Learners
• IntFormer: Predicting pedestrian intention with the aid of the Transformer architecture
• Coupling Intent and Action for Pedestrian Crossing Behavior Prediction
• Predicting Pedestrian Crossing Intention with Feature Fusion and Spatio-Temporal Attention
• Multi-Modal Hybrid Architecture for Pedestrian Action Prediction
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• RNN-based Pedestrian Crossing Prediction using Activity and Pose-related\n Features
• Pedestrian Action Anticipation using Contextual Feature Fusion in Stacked RNNs
• Is the Pedestrian going to Cross? Answering by 2D Pose Estimation
• Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty
• Attention Is All You Need
• Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
• Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
• Effective Approaches to Attention-based Neural Machine Translation

Related

• A low complexity contextual stacked ensemble-learning approach for pedestrian intent prediction
• Predicting Pedestrian Crossing Intention with Feature Fusion and Spatio-Temporal Attention
• Learning Trajectory-Conditioned Relations to Predict Pedestrian Crossing Behavior
• Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction
• Pedestrian Intention Prediction via Vision-Language Foundation Models