Incremental Human-Object Interaction Detection with Invariant Relation Representation Learning

2025/10/30 by Yana Wei, Zeen Chi, Wei, Yana +11
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.2510.27020

openalex publication_date 2025/10/30 · openalex created_date 2025/11/05 · openalex updated_date 2026/07/28

Abstract

In open-world environments, human-object interactions (HOIs) evolve continuously, challenging conventional closed-world HOI detection models. Inspired by humans' ability to progressively acquire knowledge, we explore incremental HOI detection (IHOID) to develop agents capable of discerning human-object relations in such dynamic environments. This setup confronts not only the common issue of catastrophic forgetting in incremental learning but also distinct challenges posed by interaction drift and detecting zero-shot HOI combinations with sequentially arriving data. Therefore, we propose a novel exemplar-free incremental relation distillation (IRD) framework. IRD decouples the learning of objects and relations, and introduces two unique distillation losses for learning invariant relation features across different HOI combinations that share the same relation. Extensive experiments on HICO-DET and V-COCO datasets demonstrate the superiority of our method over state-of-the-art baselines in mitigating forgetting, strengthening robustness against interaction drift, and generalization on zero-shot HOIs. Code is available at \hrefhttps://github.com/weiyana/ContinualHOIthis HTTP URL

Citations

• Interactive Continual Learning Architecture for Long-Term Personalization of Home Service Robots
• A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Tasks
• A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm
• HOI4ABOT: Human-Object Interaction Anticipation for Human Intention Reading Collaborative roBOTs
• FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning
• RLIPv2: Fast Scaling of Relational Language-Image Pre-training
• Exploring Predicate Visual Context in Detecting Human-Object Interactions
• Continual Detection Transformer for Incremental Object Detection
• HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Models
• Prototype-Sample Relation Distillation: Towards Replay-Free Continual Learning
• Dense Network Expansion for Class Incremental Learning
• Unified Visual Relationship Detection with Vision and Language Models
• EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularization
• Error Sensitivity Modulation based Experience Replay: Mitigating Abrupt Representation Drift in Continual Learning
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• ALIFE: Adaptive Logit Regularizer and Feature Replay for Incremental Semantic Segmentation
• Few-Shot Continual Active Learning by a Robot
• RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detection
• DETRs with Hybrid Matching
• General Incremental Learning with Domain-aware Categorical Representations
• Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillation
• Discovering Human-Object Interaction Concepts via Self-Compositional Learning
• GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection
• Efficient Two-Stage Detection of Human-Object Interactions with a Novel Unary-Pairwise Transformer
• Incremental Learning in Semantic Segmentation from Image Labels
• DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion
• Continual Learning via Bit-Level Information Preserving
• Emerging Properties in Self-Supervised Vision Transformers
• Affordance Transfer Learning for Human-Object Interaction Detection
• Rainbow Memory: Continual Learning with a Memory of Diverse Samples
• Detecting Human-Object Interaction via Fabricated Compositional Learning
• Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations
• Reformulating HOI Detection as Adaptive Set Prediction
• QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information
• Learning Transferable Visual Models From Natural Language Supervision
• HOI Analysis: Integrating and Decomposing Human-Object Interaction
• Contextual Heterogeneous Graph Network for Human-Object Interaction Detection
• Visual Compositional Learning for Human-Object Interaction Detection
• Momentum Contrast for Unsupervised Visual Representation Learning
• Large Scale Incremental Learning
• iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection
• Detecting and Recognizing Human-Object Interactions
• Learning to Detect Human-Object Interactions
• iCaRL: Incremental Classifier and Representation Learning
• Learning without Forgetting
• Deep Residual Learning for Image Recognition
• Visual Semantic Role Labeling
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Related

• RR-Net: Injecting Interactive Semantics in Human-Object Interaction Detection
• Cascaded Human-Object Interaction Recognition
• Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Models
• Funnel-HOI: Top-Down Perception for Zero-Shot HOI Detection
• Transferable Interactiveness Knowledge for Human-Object Interaction Detection