Learning Representations of Sets through Optimized Permutations

2018/12/10 by Zhang, Yan, Hare, Jonathon, Prügel-Bennett, Adam · 1 citation
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1812.03928

Abstract

Representations of sets are challenging to learn because operations on sets should be permutation-invariant. To this end, we propose a Permutation-Optimisation module that learns how to permute a set end-to-end. The permuted set can be further processed to learn a permutation-invariant representation of that set, avoiding a bottleneck in traditional set models. We demonstrate our model's ability to learn permutations and set representations with either explicit or implicit supervision on four datasets, on which we achieve state-of-the-art results: number sorting, image mosaics, classification from image mosaics, and visual question answering.

Cited by

• Adaptive Canonicalization with Application to Invariant Anisotropic Geometric Networks

Related

• Learning Functions over Sets via Permutation Adversarial Networks
• Representation Learning with Multisets
• FSPool: Learning Set Representations with Featurewise Sort Pooling
• Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
• Set Interdependence Transformer: Set-to-Sequence Neural Networks for Permutation Learning and Structure Prediction