The general theory of permutation equivarant neural networks and higher order graph variational encoders

2020/04/08 by Erik H. Thiede, Erik Henning Thiede, Truong Son Hy +4 · 2 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · Mathematics · #Advanced Graph Neural Networks #Bioinformatics and Genomic Networks #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Topic Modeling #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2004.03990

arxiv created 2020/04/08 · openalex publication_date 2020/04/08 · arxiv updated 2020/04/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Previous work on symmetric group equivariant neural networks generally only considered the case where the group acts by permuting the elements of a single vector. In this paper we derive formulae for general permutation equivariant layers, including the case where the layer acts on matrices by permuting their rows and columns simultaneously. This case arises naturally in graph learning and relation learning applications. As a specific case of higher order permutation equivariant networks, we present a second order graph variational encoder, and show that the latent distribution of equivariant generative models must be exchangeable. We demonstrate the efficacy of this architecture on the tasks of link prediction in citation graphs and molecular graph generation.

Citations

• Backpropagation Applied to Handwritten Zip Code Recognition
• Collective Classification in Network Data
• Variational Graph Auto-Encoders
• Permutation-equivariant neural networks applied to dynamics prediction
• Equivariance Through Parameter-Sharing
• Neural Message Passing for Quantum Chemistry
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• Invariant and Equivariant Graph Networks
• Universal approximations of permutation invariant/equivariant functions by deep neural networks
• Graph Star Net for Generalized Multi-Task Learning

Cited by

• Adaptive Canonicalization with Application to Invariant Anisotropic Geometric Networks
• A multi-dimensional quantum estimation and model learning framework based on variational Bayesian inference

Related

• E(n) Equivariant Graph Neural Networks
• Subgraph Permutation Equivariant Networks
• SpeqNets: Sparsity-aware Permutation-equivariant Graph Networks
• Graph Automorphism Group Equivariant Neural Networks
• Permutation Equivariant Generative Adversarial Networks for Graphs