Maxout Polytopes

2025/09/25 by Balakin, Andrei, Cox, Shelby, Loho, Georg +1
#Combinatorics (math.CO) #Discrete Mathematics (cs.DM) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2509.21286

Abstract

Maxout polytopes are defined by feedforward neural networks with maxout activation function and non-negative weights after the first layer. We characterize the parameter spaces and extremal f-vectors of maxout polytopes for shallow networks, and we study the separating hypersurfaces which arise when a layer is added to the network. We also show that maxout polytopes are cubical for generic networks without bottlenecks.

Citations

• Better Neural Network Expressivity: Subdividing the Simplex
• On the Depth of Monotone ReLU Neural Networks and ICNNs
• Depth-Bounds for Neural Networks via the Braid Arrangement
• On the Expressiveness of Rational ReLU Neural Networks With Bounded Depth
• Neural Networks and (Virtual) Extended Formulations
• The Real Tropical Geometry of Neural Networks
• On Minimal Depth in Neural Networks
• Revisiting Tropical Polynomial Division: Theory, Algorithms and Application to Neural Networks
• When Deep Learning Meets Polyhedral Theory: A Survey
• Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes
• A Combinatorial Perspective on the Optimization of Shallow ReLU Networks
• Algorithmic Determination of the Combinatorial Structure of the Linear Regions of ReLU Neural Networks
• Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums
• Bounding and Counting Linear Regions of Deep Neural Networks
• Input Convex Neural Networks
• On the Number of Linear Regions of Deep Neural Networks
• Construction techniques for cubical complexes, odd cubical 4-polytopes, and prescribed dual manifolds
• Neighborly cubical polytopes

Related

• On the expressivity of sparse maxout networks
• Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums
• Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes
• Expected Gradients of Maxout Networks and Consequences to Parameter Initialization
• Neural Networks and (Virtual) Extended Formulations