Evolution through Large Models

2022/06/17 by Joel Lehman, Lehman, Joel, Jonathan Gordon +9 · 2 voices · 28 citations
Computer Science · #Artificial intelligence #Code (set theory) #Computer science #Context (archaeology) #Data science #Domain (mathematical analysis) #Evolutionary Algorithms and Applications #Genetic programming #Machine Learning and Algorithms #Machine learning #Programming language #Python (programming language) #Reinforcement Learning in Robotics #Reinforcement learning #Terrain #cs.NE

paper · pdf · doi:10.48550/arxiv.2206.08896

published in arXiv (Cornell University) (Cornell University)

arxiv created 2022/06/17 · openalex publication_date 2022/06/17 · arxiv updated 2022/06/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

This paper pursues the insight that large language models (LLMs) trained to generate code can vastly improve the effectiveness of mutation operators applied to programs in genetic programming (GP). Because such LLMs benefit from training data that includes sequential changes and modifications, they can approximate likely changes that humans would make. To highlight the breadth of implications of such evolution through large models (ELM), in the main experiment ELM combined with MAP-Elites generates hundreds of thousands of functional examples of Python programs that output working ambulating robots in the Sodarace domain, which the original LLM had never seen in pre-training. These examples then help to bootstrap training a new conditional language model that can output the right walker for a particular terrain. The ability to bootstrap new models that can output appropriate artifacts for a given context in a domain where zero training data was previously available carries implications for open-endedness, deep learning, and reinforcement learning. These implications are explored here in depth in the hope of inspiring new directions of research now opened up by ELM.

Cited by

• Autonomous Discovery of Wireless Communications Algorithms
• MEMENTO: Memory-Guided Memetic Code-as-Policy Evolution
• Informing Acquisition Functions via Foundation Models for Molecular Discovery
• CogMCTS: A Novel Cognitive-Guided Monte Carlo Tree Search Framework for Iterative Heuristic Evolution with Large Language Models
• MultiGA: Leveraging Multi-Source Seeding in Genetic Algorithms
• Scaling Multi-Agent Environment Co-Design with Diffusion Models
• An In-depth Study of LLM Contributions to the Bin Packing Problem
• Autogenic transitions in individuality
• FELA: A Multi-Agent Evolutionary System for Feature Engineering of Industrial Event Log Data
• Accelerating Materials Design via LLM-Guided Evolutionary Search
• Combining Large Language Models and Gradient-Free Optimization for Automatic Control Policy Synthesis
• Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs
• Large Language Models in Operations Research: Methods, Applications, and Challenges
• Mimicking the Physicist's Eye:A VLM-centric Approach for Physics Formula Discovery
• Generative Data Refinement: Just Ask for Better Data
• \(X\)-evolve: Solution space evolution powered by large language models
• Towards Execution-Grounded Automated AI Research
• How Should We Meta-Learn Reinforcement Learning Algorithms?
• Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI
• AlphaEvolve: A coding agent for scientific and algorithmic discovery
• ELFuzz: Efficient Input Generation via LLM-driven Synthesis Over Fuzzer Space
• EvoGit: Decentralized Code Evolution via Git-Based Multi-Agent Collaboration
• RoboMoRe: LLM-based Robot Co-design via Joint Optimization of Morphology and Reward
• EVOREFUSE: Evolutionary Prompt Optimization for Evaluation and Mitigation of LLM Over-Refusal to Pseudo-Malicious Instructions
• RedAHD: Reduction-Based End-to-End Automatic Heuristic Design with Large Language Models
• InnateCoder: Learning Programmatic Options with Foundation Models
• OR-Agent: Bridging Evolutionary Search and Structured Research for Automated Algorithm Discovery
• Algorithm Discovery With LLMs: Evolutionary Search Meets Reinforcement Learning

Discussions

• Evolution Through Large Models [hn, 32 points, 14 comments]
• Evolution Through Large Models [hn, 3 points, 0 comments]

Related

• Evolving Code with A Large Language Model
• Large Language Models As Evolution Strategies
• Exploring the Improvement of Evolutionary Computation via Large Language Models
• Controlling the Mutation in Large Language Models for the Efficient Evolution of Algorithms
• Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning