HATSolver: Learning Groebner Bases with Hierarchical Attention Transformers

2025/12/09 by Malhou, Mohamed, Perret, Ludovic, Lauter, Kristin
Computer Science · Physics and Astronomy · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Numerical Methods and Algorithms #Polynomial and algebraic computation

paper · doi:10.48550/arxiv.2512.14722

openalex publication_date 2025/12/09 · openalex created_date 2025/12/19 · openalex updated_date 2026/07/28

Abstract

At NeurIPS 2024, Kera et al. introduced the use of transformers for computing Groebner bases, a central object in computer algebra with numerous practical applications. In this paper, we improve this approach by applying Hierarchical Attention Transformers (HATs) to solve systems of multivariate polynomial equations via Groebner bases computation. The HAT architecture incorporates a tree-structured inductive bias that enables the modeling of hierarchical relationships present in the data and thus achieves significant computational savings compared to conventional flat attention models. We generalize to arbitrary depths and include a detailed computational cost analysis. Combined with curriculum learning, our method solves instances that are much larger than those in Kera et al. (2024 Learning to compute Groebner bases)

Citations

• From Bytes to Ideas: Language Modeling with Autoregressive U-Nets
• Computational Algebra with Attention: Transformer Oracles for Border Basis Algorithms
• Tropical Attention: Neural Algorithmic Reasoning for Combinatorial Algorithms
• Geometric Generality of Transformer-Based Gröbner Basis Computation
• Block Transformer: Global-to-Local Language Modeling for Fast Inference
• SpaceByte: Towards Deleting Tokenization from Large Language Modeling
• Neural Algorithmic Reasoning for Combinatorial Optimisation
• MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers
• An Exploration of Hierarchical Attention Transformers for Efficient Long Document Classification
• ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention
• Hi-Transformer: Hierarchical Interactive Transformer for Efficient and Effective Long Document Modeling
• Learning selection strategies in Buchberger's algorithm
• Neural Execution of Graph Algorithms
• Neural Legal Judgment Prediction in English
• Attention Is All You Need
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Related

• Discovering Hidden Algebraic Structures via Transformers with Rank-Aware Beam GRPO
• Learning to Compute Gröbner Bases
• Signature-based algorithms to compute Groebner bases
• Computational Algebra with Attention: Transformer Oracles for Border Basis Algorithms
• Involutive Algorithms for Computing Groebner Bases