Müntz-Szász Networks: Neural Architectures with Learnable Power-Law Bases

2025/12/22 by Gnankan Landry Regis N'guessan, N'guessan, Gnankan Landry Regis
Materials Science · Physics and Astronomy · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Materials Science #Model Reduction and Neural Networks #Quantum many-body systems

paper · doi:10.48550/arxiv.2512.22222

openalex publication_date 2025/12/22 · openalex created_date 2025/12/31 · openalex updated_date 2026/07/28

Abstract

Standard neural network architectures employ fixed activation functions (ReLU, tanh, sigmoid) that are poorly suited for approximating functions with singular or fractional power behavior, a structure that arises ubiquitously in physics, including boundary layers, fracture mechanics, and corner singularities. We introduce Müntz-Szász Networks (MSN), a novel architecture that replaces fixed smooth activations with learnable fractional power bases grounded in classical approximation theory. Each MSN edge computes ϕ(x) = ∑_k a_k |x|^μ_k + ∑_k b_k sign(x)|x|^λ_k, where the exponents \μ_k, λ_k\ are learned alongside the coefficients. We prove that MSN inherits universal approximation from the Müntz-Szász theorem and establish novel approximation rates: for functions of the form |x|^α, MSN achieves error O(|μ- α|²) with a single learned exponent, whereas standard MLPs require O(ε^-1/α) neurons for comparable accuracy. On supervised regression with singular target functions, MSN achieves 5-8x lower error than MLPs with 10x fewer parameters. Physics-informed neural networks (PINNs) represent a particularly demanding application for singular function approximation; on PINN benchmarks including a singular ODE and stiff boundary-layer problems, MSN achieves 3-6x improvement while learning interpretable exponents that match the known solution structure. Our results demonstrate that theory-guided architectural design can yield dramatic improvements for scientifically-motivated function classes.

Citations

• KAN: Kolmogorov-Arnold Networks
• Neural Network Approximation
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
• Implicit Neural Representations with Periodic Activation Functions
• Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems
• Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Error bounds for approximations with deep ReLU networks
• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Related

• Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next
• Brain-Inspired Physics-Informed Neural Networks: Bare-Minimum Neural Architectures for PDE Solvers
• Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next
• Physics-Informed Neural Networks: Minimizing Residual Loss with Wide Networks and Effective Activations
• KAN: Kolmogorov-Arnold Networks