2021/03/29 by Jinwoo Kim, Jaehoon Yoo, Kim, Jinwoo +5 · 17 citations
Computer Science · Mathematics · #Artificial intelligence #Autoencoder #Cardinality (data modeling) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Data mining #Deep learning #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Generative grammar #Generative model #Invariant (physics) #Machine Learning (cs.LG) #Mathematics #Partition (number theory) #Permutation (music) #Set (abstract data type) #Theoretical computer science #cs.CV #cs.LG
paper · pdf · doi:10.48550/arxiv.2103.15619
published in arXiv (Cornell University) (Cornell University) · 19 pages, 20 figures
arxiv created 2021/03/29 · openalex publication_date 2021/03/29 · arxiv updated 2021/03/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Generative modeling of set-structured data, such as point clouds, requires reasoning over local and global structures at various scales. However, adopting multi-scale frameworks for ordinary sequential data to a set-structured data is nontrivial as it should be invariant to the permutation of its elements. In this paper, we propose SetVAE, a hierarchical variational autoencoder for sets. Motivated by recent progress in set encoding, we build SetVAE upon attentive modules that first partition the set and project the partition back to the original cardinality. Exploiting this module, our hierarchical VAE learns latent variables at multiple scales, capturing coarse-to-fine dependency of the set elements while achieving permutation invariance. We evaluate our model on point cloud generation task and achieve competitive performance to the prior arts with substantially smaller model capacity. We qualitatively demonstrate that our model generalizes to unseen set sizes and learns interesting subset relations without supervision. Our implementation is available at https://github.com/jw9730/setvae.