Open-Endedness is Essential for Artificial Superhuman Intelligence

2024/06/06 by Edward Hughes, Michael Dennis, Hughes, Edward +15 · 6 voices · 34 citations
Computer Science · Physics and Astronomy · #Computability, Logic, AI Algorithms #Space Science and Extraterrestrial Life #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2406.04268

openalex publication_date 2024/06/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In recent years there has been a tremendous surge in the general capabilities of AI systems, mainly fuelled by training foundation models on internetscale data. Nevertheless, the creation of openended, ever self-improving AI remains elusive. In this position paper, we argue that the ingredients are now in place to achieve openendedness in AI systems with respect to a human observer. Furthermore, we claim that such open-endedness is an essential property of any artificial superhuman intelligence (ASI). We begin by providing a concrete formal definition of open-endedness through the lens of novelty and learnability. We then illustrate a path towards ASI via open-ended systems built on top of foundation models, capable of making novel, humanrelevant discoveries. We conclude by examining the safety implications of generally-capable openended AI. We expect that open-ended foundation models will prove to be an increasingly fertile and safety-critical area of research in the near future.

Cited by

• The World Is Bigger! A Computationally-Embedded Perspective on the Big World Hypothesis
• Solipsistic Superintelligence is Unlikely to be Cooperative
• Towards Open-Ended Visual Scientific Discovery with Sparse Autoencoders
• On Improvisation and Open-Endedness: Insights for Experiential AI
• Cultural Alien Sampler: Open-ended art generation balancing originality and coherence
• The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
• PaperSearchQA: Learning to Search and Reason over Scientific Papers with RLVR
• Guiding Evolution of Artificial Life Using Vision-Language Models
• Policy Compatible Skill Incremental Learning via Lazy Learning Interface
• Benchmarking Open-Ended Multi-Agent Coordination in Language Agents
• Towards Open-Ended Discovery for Low-Resource NLP
• The Future of Artificial Intelligence in Ecosystem Modeling
• Combinatorial Creativity: A New Frontier in Generalization Abilities
• To Trade or Not to Trade: An Agentic Approach to Estimating Market Risk Improves Trading Decisions
• VoyagerVision: Investigating the Role of Multi-modal Information for Open-ended Learning Systems
• Curious Causality-Seeking Agents Learn Meta Causal World
• Efficient Skill Discovery via Regret-Aware Optimization
• Causal-Paced Deep Reinforcement Learning
• TRACED: Transition-aware Regret Approximation with Co-learnability for Environment Design
• HeuriGym: An Agentic Benchmark for LLM-Crafted Heuristics in Combinatorial Optimization
• Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models
• VITA: Zero-Shot Value Functions via Test-Time Adaptation of Vision-Language Models
• Cross-Entropy Games for Language Models: From Implicit Knowledge to General Capability Measures
• Truly Self-Improving Agents Require Intrinsic Metacognitive Learning
• From Connectivity to Autonomy: The Dawn of Self-Evolving Communication Systems
• ROTATE: Regret-driven Open-ended Training for Ad Hoc Teamwork
• OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Models
• Scalable Strategies for Continual Learning with Replay
• Opening the Scope of Openness in AI
• Group-Evolving Agents: Open-Ended Self-Improvement via Experience Sharing
• Scale-Dependent Collective Adaptation in Self-Amending LLM Societies: A Cross-Family Study of Emergent Governance
• Improving Zero-Shot Offline RL via Behavioral Task Sampling
• Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMs
• A Self-Improving Coding Agent

Discussions

• Open-Endedness Is Essential for Artificial Superhuman Intelligence [hn, 7 points, 0 comments]
• Open-Endedness Is Essential for Artificial Superhuman Intelligence [hn, 5 points, 0 comments]
• In this paper, researchers from Google Deepmind argue that the ability to constantly improve oneself is an essential characteristic of artificial superintelligence (ASI). Open-Endedness is Essential [bsky, 1 points, 0 comments]
• 1: open-ended means that it will keep producing novel and learnable artifacts (see the definition here: arxiv.org/abs/2406.04268), on the timescale of interest for the observer. 2: I think as a thoug [bsky, 1 points, 1 comments]
• novelty = f(audience, goal)? there's a nice exposition of this topic in @rockt.ai and others' position paper arxiv.org/abs/2406.042... which, to prove a meta-point, delivers a rather "obvious" message [bsky, 1 points, 0 comments]
• Google DeepMind on the necessity of open-ended systems in order to achieve ASI [bsky, 0 points, 0 comments]

Related

• Safety Must Precede the Deployment of Open-Ended AI
• Open Questions in Creating Safe Open-ended AI: Tensions Between Control and Creativity
• Open Ended Intelligence: The individuation of Intelligent Agents
• AI for the Open-World: the Learning Principles
• Why Open-Endedness Matters