2025/03/15 by Zhihao Zhu, Zhu, Zhihao
Computer Science · Engineering · #Artificial Intelligence in Games #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Motion and Animation #Machine Learning (cs.LG) #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2503.14524
openalex publication_date 2025/03/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Representing a dynamic scene using a structured spatial-temporal scene graph is a novel and particularly challenging task. To tackle this task, it is crucial to learn the temporal interactions between objects in addition to their spatial relations. Due to the lack of explicitly annotated temporal relations in current benchmark datasets, most of the existing spatial-temporal scene graph generation methods build dense and abstract temporal connections among all objects across frames. However, not all temporal connections are encoding meaningful temporal dynamics. We propose a novel spatial-temporal scene graph generation method that selectively builds temporal connections only between temporal-relevant objects pairs and represents the temporal relations as explicit edges in the scene graph. The resulting sparse and explicit temporal representation allows us to improve upon strong scene graph generation baselines by up to 4.4% in Scene Graph Detection. In addition, we show that our approach can be leveraged to improve downstream vision tasks. Particularly, applying our approach to action recognition, shows 0.6% gain in mAP in comparison to the state-of-the-art