Representation Learning with Multisets

2019/11/19 by Vasco Portilheiro, Portilheiro, Vasco
Computer Science · #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.1911.08577

openalex publication_date 2019/11/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We study the problem of learning permutation invariant representations that can capture "flexible" notions of containment. We formalize this problem via a measure theoretic definition of multisets, and obtain a theoretically-motivated learning model. We propose training this model on a novel task: predicting the size of the symmetric difference (or intersection) between pairs of multisets. We demonstrate that our model not only performs very well on predicting containment relations (and more effectively predicts the sizes of symmetric differences and intersections than DeepSets-based approaches with unconstrained object representations), but that it also learns meaningful representations.

Citations

• Gradient-based learning applied to document recognition
• Deep Models of Interactions Across Sets

Related

• Learning Representations of Sets through Optimized Permutations
• Set Representation Learning with Generalized Sliced-Wasserstein Embeddings
• Learning Functions on Multiple Sets using Multi-Set Transformers
• FSPool: Learning Set Representations with Featurewise Sort Pooling
• Joint Learning of Set Cardinality and State Distribution