Detecting Human-Object Interaction via Fabricated Compositional Learning

2021/03/15 by Zhi Hou, Hou, Zhi, Baosheng Yu +7 · 14 citations
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #cs.CV

paper · pdf · doi:10.48550/arxiv.2103.08214

Accepted to CVPR2021; update code, figures, appendix(Object Detector Analysis)

arxiv created 2021/03/25 · arxiv updated 2021/03/26

Abstract

Human-Object Interaction (HOI) detection, inferring the relationships between human and objects from images/videos, is a fundamental task for high-level scene understanding. However, HOI detection usually suffers from the open long-tailed nature of interactions with objects, while human has extremely powerful compositional perception ability to cognize rare or unseen HOI samples. Inspired by this, we devise a novel HOI compositional learning framework, termed as Fabricated Compositional Learning (FCL), to address the problem of open long-tailed HOI detection. Specifically, we introduce an object fabricator to generate effective object representations, and then combine verbs and fabricated objects to compose new HOI samples. With the proposed object fabricator, we are able to generate large-scale HOI samples for rare and unseen categories to alleviate the open long-tailed issues in HOI detection. Extensive experiments on the most popular HOI detection dataset, HICO-DET, demonstrate the effectiveness of the proposed method for imbalanced HOI detection and significantly improve the state-of-the-art performance on rare and unseen HOI categories. Code is available at https://github.com/zhihou7/HOI-CL.

Cited by

• Incremental Human-Object Interaction Detection with Invariant Relation Representation Learning
• Visual Diversity and Region-aware Prompt Learning for Zero-shot HOI Detection
• Learning by Imagining: Debiased Feature Augmentation for Compositional Zero-Shot Learning
• DQEN: Dual Query Enhancement Network for DETR-based HOI Detection
• Open-Vocabulary HOI Detection with Interaction-aware Prompt and Concept Calibration
• HOLa: Zero-Shot HOI Detection with Low-Rank Decomposed VLM Feature Adaptation
• From Semantics, Scene to Instance-awareness: Distilling Foundation Model for Grounded Open-vocabulary Situation Recognition
• Dynamic Scoring with Enhanced Semantics for Training-Free Human-Object Interaction Detection
• Funnel-HOI: Top-Down Perception for Zero-Shot HOI Detection
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection
• Bilateral Collaboration with Large Vision-Language Models for Open Vocabulary Human-Object Interaction Detection
• Locality-Aware Zero-Shot Human-Object Interaction Detection
• Foundation Model-Driven Framework for Human-Object Interaction Prediction with Segmentation Mask Integration
• Unveiling Hidden Collaboration within Mixture-of-Experts in Large Language Models

Related

• Visual Compositional Learning for Human-Object Interaction Detection
• Compositional Learning in Transformer-Based Human-Object Interaction Detection
• Discovering Human-Object Interaction Concepts via Self-Compositional Learning
• Weakly-supervised HOI Detection via Prior-guided Bi-level Representation Learning
• Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Models