Image Scene Graph Generation (SGG) Benchmark

2021/07/27 by Xiaotian Han, Han, Xiaotian, Jianwei Yang +9
Computer Science · #Advanced Image and Video Retrieval Techniques #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.2107.12604

openalex publication_date 2021/07/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

There is a surge of interest in image scene graph generation (object, attribute and relationship detection) due to the need of building fine-grained image understanding models that go beyond object detection. Due to the lack of a good benchmark, the reported results of different scene graph generation models are not directly comparable, impeding the research progress. We have developed a much-needed scene graph generation benchmark based on the maskrcnn-benchmark and several popular models. This paper presents main features of our benchmark and a comprehensive ablation study of scene graph generation models using the Visual Genome and OpenImages Visual relationship detection datasets. Our codebase is made publicly available at https://github.com/microsoft/scene_graph_benchmark.

Citations

• Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
• BLOCK: Bilinear Superdiagonal Fusion for Visual Question Answering and\n Visual Relationship Detection
• Action Genome: Actions as Composition of Spatio-temporal Scene Graphs

Related

• Scene Graph Generation: A Comprehensive Survey
• Scene Graph Generation with External Knowledge and Image Reconstruction
• Fine-Grained is Too Coarse: A Novel Data-Centric Approach for Efficient Scene Graph Generation
• An Interpretable Model for Scene Graph Generation
• RepSGG: Novel Representations of Entities and Relationships for Scene Graph Generation