Robust Autonomy Emerges from Self-Play

2025/02/05 by Marco Cusumano-Towner, Cusumano-Towner, Marco, David Hafner +22 · 19 voices · 21 citations
Social Sciences · #Evolutionary Game Theory and Cooperation #Experimental Behavioral Economics Studies #cs.AI #cs.LG #cs.RO

paper · pdf · doi:10.48550/arxiv.2502.03349

openalex publication_date 2025/02/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Self-play has powered breakthroughs in two-player and multi-player games. Here we show that self-play is a surprisingly effective strategy in another domain. We show that robust and naturalistic driving emerges entirely from self-play in simulation at unprecedented scale -- 1.6~billion~km of driving. This is enabled by Gigaflow, a batched simulator that can synthesize and train on 42 years of subjective driving experience per hour on a single 8-GPU node. The resulting policy achieves state-of-the-art performance on three independent autonomous driving benchmarks. The policy outperforms the prior state of the art when tested on recorded real-world scenarios, amidst human drivers, without ever seeing human data during training. The policy is realistic when assessed against human references and achieves unprecedented robustness, averaging 17.5 years of continuous driving between incidents in simulation.

Citations

• Mastering the game of Stratego with model-free multiagent reinforcement learning

Cited by

• TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrations
• Human-like autonomy emerges from self-play and a pinch of human data
• 123D: Unifying Multi-Modal Autonomous Driving Data at Scale
• Learning to Drive in New Cities Without Human Demonstrations
• Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning
• CaRL: Learning Scalable Planning Policies with Simple Rewards
• LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving
• Pseudo-Expert Regularized Offline RL for End-to-End Autonomous Driving in Photorealistic Closed-Loop Environments
• Agile Flight Emerges from Multi-Agent Competitive Racing
• Post-Training and Test-Time Scaling of Generative Agent Behavior Models for Interactive Autonomous Driving
• Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulation
• SimScale: Learning to Drive via Real-World Simulation at Scale
• Superhuman AI for Stratego Using Self-Play Reinforcement Learning and Test-Time Search
• JaxMARL-HFT: GPU-Accelerated Large-Scale Multi-Agent Reinforcement Learning for High-Frequency Trading
• ZTRS: Zero-Imitation End-to-end Autonomous Driving with Trajectory Scoring
• SPACeR: Self-Play Anchoring with Centralized Reference Models
• Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modeling
• Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving
• RDAR: Reward-Driven Agent Relevance Estimation for Autonomous Driving
• SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer
• Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving

Discussions

• Robust autonomy emerges from self-play [hn, 140 points, 62 comments]
• I've been talking about writing this paper to anyone who would listen since 2020. I bombed a bunch of job talks trying to convince companies to work on this. It's so nice to finally just be able to sa [bsky, 92 points, 3 comments]
• Apple team shows self-driving AI can learn entirely by practicing against itself - no human driving data needed In testing, their system averages 17.5 years of continuous driving between incidents, f [bsky, 77 points, 6 comments]
• Crazily amazing work by @eugenevinitsky.bsky.social @senerozan.bsky.social & team, setting the bar so high for anyone working in autonomous driving these days. Check it out arxiv.org/abs/2502.03349 [bsky, 18 points, 0 comments]
• arxiv.org/abs/2502.03349 Awesome planning results by Vladlen Koltun's new lab (@twkillian.bsky.social, @eugenevinitsky.bsky.social @senerozan.bsky.social and others) that everyone working on driving s [bsky, 14 points, 1 comments]
• Robust autonomy emerges from self-play [bsky, 1 points, 0 comments]
• GIGAFLOWすごいと感じるのに、これだけのことができてもApple carは上手くいかなかったというのがよくわからなくなってしまう arxiv.org/abs/2502.03349 [bsky, 1 points, 0 comments]
• So apparently playing with oneself makes one more adaptable? *big brain* arxiv.org/abs/2502.03349 New record-breaking approach to training self-driving agents [bsky, 1 points, 0 comments]
• Robust autonomy emerges from self-play https://arxiv.org/abs/2502.03349 https://news.ycombinator.com/item?id=42968700 [bsky, 0 points, 0 comments]
• Robust autonomy emerges from self-play https://arxiv.org/abs/2502.03349 [bsky, 0 points, 0 comments]
• arxiv.org/abs/2502.03349 [bsky, 0 points, 0 comments]
• arxiv.org/abs/2502.03349 [bsky, 0 points, 0 comments]
• https://bsky.app/profile/news.ycombinator.com.web.brid.gy/post/3lhl6uewgwsi2 [bsky, 0 points, 0 comments]
• Robust autonomy emerges from self-play (arxiv.org) Main Link | Discussion [bsky, 0 points, 0 comments]
• 模拟环境助力自动驾驶训练突破 得克萨斯大学研究表明，通过模拟环境中的自我对弈，自动驾驶训练效率显著提升。Gigaflow模拟器在单节点上实现每小时42年驾驶经验的数据合成与训练。该策略在多个基准测试中表现出色，超越此前水平，且未用人类数据。真实场景测试中，稳健性极高，平均17.5年才发生一次事故。 #自动驾驶 #模拟训练 #人工智能 #Gigaflow [bsky, 0 points, 0 comments]
• Robust autonomy emerges from self-play https://arxiv.org/abs/2502.03349 [comments] [110 points] [bsky, 0 points, 0 comments]
• Robust autonomy emerges from self-play https://arxiv.org/abs/2502.03349 (https://news.ycombinator.com/item?id=42968700) [bsky, 0 points, 0 comments]
• Robust Autonomy Emerges from Self-Play https://arxiv.org/abs/2502.03349 (https://news.ycombinator.com/item?id=42968700) [bsky, 0 points, 0 comments]
• Robust Autonomy Emerges from Self-Play #HackerNews arxiv.org/abs/... [bsky, 0 points, 0 comments]

Related

• Human-like autonomy emerges from self-play and a pinch of human data
• Learning to Drive via Asymmetric Self-Play
• Building reliable sim driving agents by scaling self-play
• SPACeR: Self-Play Anchoring with Centralized Reference Models
• Sim-on-Wheels: Physical World in the Loop Simulation for Self-Driving