Growing Interpretable Part Graphs on ConvNets via Multi-Shot Learning

2016/11/14 by Quanshi Zhang, Ruiming Cao, Zhang, Quanshi +5 · 1 citation
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.1611.04246

openalex publication_date 2016/11/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper proposes a learning strategy that extracts object-part concepts from a pre-trained convolutional neural network (CNN), in an attempt to 1) explore explicit semantics hidden in CNN units and 2) gradually grow a semantically interpretable graphical model on the pre-trained CNN for hierarchical object understanding. Given part annotations on very few (e.g., 3-12) objects, our method mines certain latent patterns from the pre-trained CNN and associates them with different semantic parts. We use a four-layer And-Or graph to organize the mined latent patterns, so as to clarify their internal semantic hierarchy. Our method is guided by a small number of part annotations, and it achieves superior performance (about 13%-107% improvement) in part center prediction on the PASCAL VOC and ImageNet datasets.

Citations

• Gradient-based learning applied to document recognition
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Cited by

• Enhancing Pre-trained Representation Classifiability can Boost its Interpretability

Related

• Explanatory Graphs for CNNs
• Unsupervised learning of object semantic parts from internal states of CNNs by population encoding
• Interpreting CNN Knowledge via an Explanatory Graph
• Part-Stacked CNN for Fine-Grained Visual Categorization
• Interactively Transferring CNN Patterns for Part Localization