How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse

2024/04/07 by Mohamed El Amine Seddik, Suei-Wen Chen, Seddik, Mohamed El Amine +8 · 3 voices · 12 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #cs.AI #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.2404.05090

openalex publication_date 2024/04/07 · arxiv published 2024/04/07 · arxiv updated 2024/04/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The phenomenon of model collapse, introduced in (Shumailov et al., 2023), refers to the deterioration in performance that occurs when new models are trained on synthetic data generated from previously trained models. This recursive training loop makes the tails of the original distribution disappear, thereby making future-generation models forget about the initial (real) distribution. With the aim of rigorously understanding model collapse in language models, we consider in this paper a statistical model that allows us to characterize the impact of various recursive training scenarios. Specifically, we demonstrate that model collapse cannot be avoided when training solely on synthetic data. However, when mixing both real and synthetic data, we provide an estimate of a maximal amount of synthetic data below which model collapse can eventually be avoided. Our theoretical conclusions are further supported by empirical validations.

Cited by

• Selecting Language Models for Social Science: Start Small, Start Open, and Validate
• Chasing Shadows: Pitfalls in LLM Security Research
• Data Value in the Age of Scaling: Understanding LLM Scaling Dynamics Under Real-Synthetic Data Mixtures
• Why Less is More (Sometimes): A Theory of Data Curation
• When Your AI Agent Succumbs to Peer-Pressure: Studying Opinion-Change Dynamics of LLMs
• First-Extinction Law for Resampling Processes
• EnAnchored-X2X: English-Anchored Optimization for Many-to-Many Translation
• A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective
• Knowledge Collapse in LLMs: When Fluency Survives but Facts Fail under Recursive Synthetic Training
• The Anti-Ouroboros Effect: Emergent Resilience in Large Language Models from Recursive Selective Feedback
• A Penalty Goes a Long Way: Measuring Lexical Diversity in Synthetic Texts Under Prompt-Influenced Length Variations
• Synthetic Voice Data for Automatic Speech Recognition in African Languages

Discussions

• not sure if this is the paper i remember but - it can be mitigated! and i see Tim's also popping off with what he knows. [bsky, 3 points, 1 comments]
• https://arxiv.org/abs/2404.05090 interesting study of model collapse and how much synthetic data you can get away with [bsky, 0 points, 0 comments]
• * [\[2404.05090\] How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse](arxiv.org/abs/2404.050...) [bsky, 0 points, 1 comments]

Related

• A Probabilistic Perspective on Model Collapse
• Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
• Knowledge Collapse in LLMs: When Fluency Survives but Facts Fail under Recursive Synthetic Training
• Theoretical Proof that Auto-regressive Language Models Collapse when Real-world Data is a Finite Set
• Rate of Model Collapse in Recursive Training