Transcendence: Generative Models Can Outperform The Experts That Train Them

2024/06/17 by Edwin Zhang, Vincent Zhu, Zhang, Edwin +13 · 8 voices · 9 citations
#cs.LG #cs.AI

paper · pdf · doi:10.48550/arxiv.2406.11741

Abstract

Generative models are trained with the simple objective of imitating the conditional probability distribution induced by the data they are trained on. Therefore, when trained on data generated by humans, we may not expect the artificial model to outperform the humans on their original objectives. In this work, we study the phenomenon of transcendence: when a generative model achieves capabilities that surpass the abilities of the experts generating its data. We demonstrate transcendence by training an autoregressive transformer to play chess from game transcripts, and show that the trained model can sometimes achieve better performance than all players in the dataset. We theoretically prove that transcendence can be enabled by low-temperature sampling, and rigorously assess this claim experimentally. Finally, we discuss other sources of transcendence, laying the groundwork for future investigation of this phenomenon in a broader setting.

Cited by

• Understanding Reasoning from Pretraining to Post-Training
• A Taxonomy of Transcendence
• AI as Governance
• Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
• Understanding temperature tuning in energy-based models
• Weak-to-Strong Generalization under Distribution Shifts
• Out-of-distribution Tests Reveal Compositionality in Chess Transformers
• The Future of Artificial Intelligence in Ecosystem Modeling
• Limitations of refinement methods for weak to strong generalization

Discussions

• Gotta thank @nsaphra.bsky.social for writing this paper, because wow it comes in useful for internet arguments about Ted Chiang’s assertion that language models can only ever “take the average of choi [bsky, 37 points, 6 comments]
• Modern generative models are trained to imitate human experts, but can they actually beat those experts? Our new paper uses imitative chess agents to explore when a model can "transcend" its training [bsky, 20 points, 4 comments]
• Also, just to point out I borrowed the term transcend from another NeurIPS paper about a somewhat related experiment/paper: arxiv.org/abs/2406.11741 [bsky, 6 points, 0 comments]
• Transcendence: Generative Models Can Outperform the Experts That Train Them [hn, 4 points, 0 comments]
• arxiv.org/pdf/2406.11741 Idk if best/renowned, but concepts like “transcendence” give a good language to critique this imo [bsky, 2 points, 0 comments]
• Transcendence: Generative Models Can Outperform the Experts That Train Them [hn, 2 points, 0 comments]
• Transcendence: Generative Models Can Outperform the Experts That Train Them [hn, 2 points, 0 comments]
• Transcendence: Generative Models Can Outperform The Experts That Train Them arxiv.org/abs/2406.117... [bsky, 0 points, 0 comments]

Related

• A Taxonomy of Transcendence
• The Generative AI Paradox: "What It Can Create, It May Not Understand"
• Learning the Latent Rules of a Game from Data: A Chess Story
• Learning by Surprise: Adaptive Mitigation of Model Collapse in Large Language Models
• If generative AI is the answer, what is the question?