FunL2O: LLM-Guided Feature Function Design for Learning to Optimize

2026/07/29 by Bingheng Li, Junyang Cai, Yupeng Zhang +3
Computer Science · #cs.LG #cs.AI

paper · pdf

arxiv created 2026/07/29 · arxiv updated 2026/07/31

Abstract

Learning-to-optimize (L2O) methods accelerate repeated optimization by training models to predict solutions, warm starts, branching decisions, or other forms of solver guidance. A critical yet largely overlooked component of these pipelines is the feature function that maps problem instances to inputs for machine learning models. Existing L2O methods typically rely on hand-crafted features, making representation design manual and largely fixed across domains. We introduce FunL2O, the first unified framework for automating feature design through LLM-driven program evolution for L2O. In a FunSearch-style loop, an LLM proposes executable feature functions, while a fixed evaluation process retrains the original L2O model and measures downstream optimization performance. We evaluate FunL2O on linear and quadratic programming tasks involving solution prediction and warm-starting, as well as on mixed-integer optimization tasks using GNN-guided backdoor branching and Predict-and-Search. Across continuous and discrete optimization tasks and four LLMs, the evolved features consistently outperform hand-crafted representations. These results establish LLM-driven feature evolution as a general and effective approach to automating representation design in L2O.

Citations

• AlphaEvolve: A coding agent for scientific and algorithmic discovery
• FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guarantees
• When GNNs meet symmetry in ILPs: an orbit-based feature augmentation approach
• PDHG-Unrolled Learning-to-Optimize Method for Large-Scale Linear Programming
• LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics
• ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution
• Learning Backdoors for Mixed Integer Linear Programs with Contrastive Learning
• Exploring the Power of Graph Neural Networks in Solving Linear Optimization Problems
• Learning to Warm-Start Fixed-Point Optimization Algorithms
• Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineering
• M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptation
• A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases
• Learning to Optimize: A Primer and A Benchmark
• Training Stronger Baselines for Learning to Optimize
• Exact Combinatorial Optimization with Graph Convolutional Neural Networks
• Learned Optimizers that Scale and Generalize

Related

• Training Stronger Baselines for Learning to Optimize
• Make Optimization Once and for All with Fine-grained Guidance
• Learning to Optimize: A Primer and A Benchmark
• Learning to optimize: A tutorial for continuous and mixed-integer optimization
• SolverLLM: Leveraging Test-Time Scaling for Optimization Problem via LLM-Guided Search