AlphaGo Moment for Model Architecture Discovery

2025/07/24 by Yixiu Liu, Yang Nan, Liu, Yixiu +14 · 12 voices · 6 citations
Computer Science · #Model-Driven Software Engineering Techniques #Software System Performance and Reliability #Advanced Software Engineering Methodologies

paper · pdf · doi:10.48550/arxiv.2507.18074

Abstract

While AI systems demonstrate exponentially improving capabilities, the pace of AI research itself remains linearly bounded by human cognitive capacity, creating an increasingly severe development bottleneck. We present ASI-Arch, the first demonstration of Artificial Superintelligence for AI research (ASI4AI) in the critical domain of neural architecture discovery--a fully autonomous system that shatters this fundamental constraint by enabling AI to conduct its own architectural innovation. Moving beyond traditional Neural Architecture Search (NAS), which is fundamentally limited to exploring human-defined spaces, we introduce a paradigm shift from automated optimization to automated innovation. ASI-Arch can conduct end-to-end scientific research in the domain of architecture discovery, autonomously hypothesizing novel architectural concepts, implementing them as executable code, training and empirically validating their performance through rigorous experimentation and past experience. ASI-Arch conducted 1,773 autonomous experiments over 20,000 GPU hours, culminating in the discovery of 106 innovative, state-of-the-art (SOTA) linear attention architectures. Like AlphaGo's Move 37 that revealed unexpected strategic insights invisible to human players, our AI-discovered architectures demonstrate emergent design principles that systematically surpass human-designed baselines and illuminate previously unknown pathways for architectural innovation. Crucially, we establish the first empirical scaling law for scientific discovery itself--demonstrating that architectural breakthroughs can be scaled computationally, transforming research progress from a human-limited to a computation-scalable process. We provide comprehensive analysis of the emergent design patterns and autonomous research capabilities that enabled these breakthroughs, establishing a blueprint for self-accelerating AI systems.

Citations

• Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents
• Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
• Gold-medalist Performance in Solving Olympiad Geometry with AlphaGeometry2
• Jamba: A Hybrid Transformer-Mamba Language Model
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• Language Models are Few-Shot Learners
• Attention Is All You Need
• Neural Architecture Search with Reinforcement Learning
• Concrete Problems in AI Safety

Cited by

• ASI-Evolve: AI Accelerates AI
• Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledge
• AgenticSciML: Collaborative Multi-Agent Systems for Emergent Discovery in Scientific Machine Learning
• Jr. AI Scientist and Its Risk Report: Autonomous Scientific Exploration from a Baseline Paper
• Glia: A Human-Inspired AI for Automated Systems Design and Optimization
• Belief-Guided Decision Making with Uncertainty Gating in the Game of Go

Discussions

• AlphaGo Moment for Model Architecture Discovery [hn, 38 points, 7 comments]
• is this paper bullshit? they’re basically claiming recursive improvement the title is cringe af, and they’re a little sloppy (LLM as a judge, fewer ablations than ideal), but it actually seems to chec [bsky, 8 points, 2 comments]
• "ASI-Arch conducted 1,773 autonomous experiments over 20,000 GPU hours, culminating in the discovery of 106 innovative, state-of-the-art (SOTA) linear attention architectures. Like AlphaGo's Move 37.. [bsky, 2 points, 0 comments]
• Try this in other fields and it collapses. Also—I doubt there has been time for proper evaluation or critique. I wouldn’t call it a revolution is science as a whole. Still, a compelling proof-of-conce [bsky, 1 points, 0 comments]
• "Professor, without knowing precisely what the danger is, would you say it's time for our viewers to crack each other's heads open and feast on the goo inside?" [bsky, 0 points, 1 comments]
• Notes on arxiv.org/abs/2507.18074 🧵 Massive claims coming out of this paper. A framework for Artificial superintelligence by having an LLM based agent loop that automatically optimizes and discovers [bsky, 0 points, 1 comments]
• Game over man! Time to build a fire and sing a couple of camp songs. arxiv.org/abs/2507.18074 [bsky, 0 points, 0 comments]
• arxiv.org/abs/2507.18074 [bsky, 0 points, 0 comments]
• AlphaGo Moment for Model Architecture Discovery 🤔 Interesting, but I would like to see more rigorous peer review, independent validation, and real-world adoption before accepting its bold claim of "A [bsky, 0 points, 0 comments]
• ..design. By open-sourcing the framework, architectures, and cognitive traces, the authors lay the groundwork for democratizing AI-driven research and inspiring future work in autonomous scientific di [bsky, 0 points, 0 comments]
• ASI-ARCH introduces a fully autonomous AI system for neural architecture discovery—moving beyond traditional NAS by enabling AI to innovate new architectures, not just optimize existing ones. Paper: a [bsky, 0 points, 0 comments]
• 2507.18074] AlphaGo Moment for Model Architecture Discovery https://arxiv.org/abs/2507.18074 [#PostSapiens [bsky, 0 points, 0 comments]

Related

• ASI-Evolve: AI Accelerates AI
• The Role of Advanced Computer Architectures in Accelerating Artificial Intelligence Workloads
• AI-as-exploration: Navigating intelligence space
• AI-Researcher: Autonomous Scientific Innovation
• Can GPT-4 Perform Neural Architecture Search?