Equivariant neural networks and piecewise linear representation theory

2024/08/01 by Joel Gibson, Gibson, Joel, Daniel Tubbenhauer +3 · 1 voice
Computer Science · Mathematics · #68T07 #FOS: Computer and information sciences #FOS: Mathematics #Group Theory (math.GR) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Primary: 20C05 #Representation Theory (math.RT) #Secondary: 05E10 #cs.LG #math.GR #math.RT #stat.ML

paper · pdf · doi:10.48550/arxiv.2408.00949

arxiv published 2024/08/01 · arxiv updated 2024/12/20

Abstract

Equivariant neural networks are neural networks with symmetry. Motivated by the theory of group representations, we decompose the layers of an equivariant neural network into simple representations. The nonlinear activation functions lead to interesting nonlinear equivariant maps between simple representations. For example, the rectified linear unit (ReLU) gives rise to piecewise linear maps. We show that these considerations lead to a filtration of equivariant neural networks, generalizing Fourier series. This observation might provide a useful tool for interpreting equivariant neural networks.

Discussions

• Equivariant Neural Networks and Piecewise Linear Representation Theory [hn, 1 points, 0 comments]

Related

• In Search of Projectively Equivariant Networks
• A Characterization Theorem for Equivariant Networks with Point-wise Activations
• Equivariant neural networks and equivarification
• What is an equivariant neural network?
• Theoretical Aspects of Group Equivariant Neural Networks