SynDaCaTE: A Synthetic Dataset For Evaluating Part-Whole Hierarchical Inference

2025/06/21 by Levi, Jake, van der Wilk, Mark
#Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2506.17558

Abstract

Learning to infer object representations, and in particular part-whole hierarchies, has been the focus of extensive research in computer vision, in pursuit of improving data efficiency, systematic generalisation, and robustness. Models which are designed to infer part-whole hierarchies, often referred to as capsule networks, are typically trained end-to-end on supervised tasks such as object classification, in which case it is difficult to evaluate whether such a model actually learns to infer part-whole hierarchies, as claimed. To address this difficulty, we present a SYNthetic DAtaset for CApsule Testing and Evaluation, abbreviated as SynDaCaTE, and establish its utility by (1) demonstrating the precise bottleneck in a prominent existing capsule model, and (2) demonstrating that permutation-equivariant self-attention is highly effective for parts-to-wholes inference, which motivates future directions for designing effective inductive biases for computer vision.

Citations

• International AI Safety Report
• Qwen2.5 Technical Report
• Provable Compositional Generalization for Object-Centric Learning
• A ConvNet for the 2020s
• A ConvNet for the 2020s
• Zero-Shot Text-to-Image Generation
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets?
• Object-Centric Learning with Slot Attention
• Language Models are Few-Shot Learners
• ReZero is All You Need: Fast Convergence at Large Depth
• Stand-Alone Self-Attention in Vision Models
• MONet: Unsupervised Scene Decomposition and Representation
• An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution
• How Many Samples are Needed to Estimate a Convolutional or Recurrent Neural Network?
• Self-Attention with Relative Position Representations
• Dynamic Routing Between Capsules
• Attention Is All You Need
• Deep Sets
• Xception: Deep Learning with Depthwise Separable Convolutions
• Deep Residual Learning for Image Recognition
• Train faster, generalize better: Stability of stochastic gradient descent
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Related

• Learning with Capsules: A Survey
• Inference for Generative Capsule Models
• Stacked Capsule Autoencoders
• United We Learn Better: Harvesting Learning Improvements From Class Hierarchies Across Tasks
• Part-Stacked CNN for Fine-Grained Visual Categorization