Unsupervised Part Discovery from Contrastive Reconstruction

2021/11/11 by Subhabrata Choudhury, Iro Laina, Choudhury, Subhabrata +5 · 6 citations
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.2111.06349

NeurIPS 2021. Project page: https://www.robots.ox.ac.uk/~vgg/research/unsup-parts/

openalex publication_date 2021/11/11 · arxiv created 2022/03/21 · arxiv updated 2022/03/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The goal of self-supervised visual representation learning is to learn strong, transferable image representations, with the majority of research focusing on object or scene level. On the other hand, representation learning at part level has received significantly less attention. In this paper, we propose an unsupervised approach to object part discovery and segmentation and make three contributions. First, we construct a proxy task through a set of objectives that encourages the model to learn a meaningful decomposition of the image into its parts. Secondly, prior work argues for reconstructing or clustering pre-computed features as a proxy to parts; we show empirically that this alone is unlikely to find meaningful parts; mainly because of their low resolution and the tendency of classification networks to spatially smear out information. We suggest that image reconstruction at the level of pixels can alleviate this problem, acting as a complementary cue. Lastly, we show that the standard evaluation based on keypoint regression does not correlate well with segmentation quality and thus introduce different metrics, NMI and ARI, that better characterize the decomposition of objects into parts. Our method yields semantic parts which are consistent across fine-grained but visually distinct categories, outperforming the state of the art on three benchmark datasets. Code is available at the project page: https://www.robots.ox.ac.uk/~vgg/research/unsup-parts/.

Citations

• Auto-Encoding Variational Bayes
• Unsupervised learning of object semantic parts from internal states of CNNs by population encoding
• Learning deep representations by mutual information estimation and maximization
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• SCALOR: Generative World Models with Scalable Object Representations
• Momentum Contrast for Unsupervised Visual Representation Learning
• Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery

Cited by

• Multi-Part Object Representations via Graph Structures and Co-Part Discovery
• PartCo: Part-Level Correspondence Priors Enhance Category Discovery
• GECO: Geometrically Consistent Embedding with Lightspeed Inference
• Unsupervised Part Discovery via Descriptor-Based Masked Image Restoration with Optimized Constraints
• Unsupervised Monocular 3D Keypoint Discovery from Multi-View Diffusion Priors
• Adaptive Part Learning for Fine-Grained Generalized Category Discovery: A Plug-and-Play Enhancement

Related

• Unsupervised Part Segmentation through Disentangling Appearance and Shape
• PARTICLE: Part Discovery and Contrastive Learning for Fine-grained Recognition
• Unsupervised Part Discovery by Unsupervised Disentanglement
• Self-Supervised Learning of Object Parts for Semantic Segmentation
• Towards Open-World Segmentation of Parts