Equivariant neural networks and piecewise linear representation theory

2024/08/01 by Gibson, Joel, Tubbenhauer, Daniel, Williamson, Geordie
#68T07 #FOS: Computer and information sciences #FOS: Mathematics #Group Theory (math.GR) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Primary: 20C05 #Representation Theory (math.RT) #Secondary: 05E10

paper · doi:10.48550/arxiv.2408.00949

Abstract

Equivariant neural networks are neural networks with symmetry. Motivated by the theory of group representations, we decompose the layers of an equivariant neural network into simple representations. The nonlinear activation functions lead to interesting nonlinear equivariant maps between simple representations. For example, the rectified linear unit (ReLU) gives rise to piecewise linear maps. We show that these considerations lead to a filtration of equivariant neural networks, generalizing Fourier series. This observation might provide a useful tool for interpreting equivariant neural networks.

Related

• In Search of Projectively Equivariant Networks
• A Characterization Theorem for Equivariant Networks with Point-wise Activations
• Equivariant neural networks and equivarification
• What is an equivariant neural network?
• Theoretical Aspects of Group Equivariant Neural Networks