On the Geometry and Optimization of Polynomial Convolutional Networks

2024/10/01 by Vahid Shahverdi, Shahverdi, Vahid, Giovanni Luca Marchetti +3 · 3 citations
Computer Science · Mathematics · #Algebraic Geometry (math.AG) #FOS: Computer and information sciences #FOS: Mathematics #Graph theory and applications #Machine Learning (cs.LG) #Matrix Theory and Algorithms #Polynomial and algebraic computation

paper · pdf · doi:10.48550/arxiv.2410.00722

openalex publication_date 2024/10/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We study convolutional neural networks with monomial activation functions. Specifically, we prove that their parameterization map is regular and is an isomorphism almost everywhere, up to rescaling the filters. By leveraging on tools from algebraic geometry, we explore the geometric properties of the image in function space of this map - typically referred to as neuromanifold. In particular, we compute the dimension and the degree of the neuromanifold, which measure the expressivity of the model, and describe its singularities. Moreover, for a generic large dataset, we derive an explicit formula that quantifies the number of critical points arising in the optimization of a regression loss.

Cited by

• Critical Points of Degenerate Metrics on Algebraic Varieties: A Tale of Overparametrization
• The Alexander-Hirschowitz theorem for neurovarieties
• Identifiable Equivariant Networks are Layerwise Equivariant

Related

• Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networks
• Algebraic Complexity and Neurovariety of Linear Convolutional Networks
• Geometry of Linear Convolutional Networks
• Geometry and Optimization of Shallow Polynomial Networks
• Activation degree thresholds and expressiveness of polynomial neural networks