Dynamic / ME-JEPA v2.0.0-rc1: Audited World-Model Runtime and Verified Training-Corpus Artifact

Recursive training on model-generated data causes generative models to lose information about rare events and eventually collapse into inaccurate outputs.

2023/05/27 by Ilia Shumailov, Shumailov, Ilia, Zakhar Shumaylov +9 · 59 voices · 158 citations
Computer Science · Psychology · #Artificial intelligence #Cognitive science #Computational Physics and Python Applications #Computer science #Curse #Data science #Epistemology #Generative grammar #Generative model #Intuition #Natural Language Processing Techniques #Phenomenon #Psychology #Sociology #The Internet #Topic Modeling #World Wide Web

paper · pdf · doi:10.48550/arxiv.2305.17493

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2023/05/27 · openalex created_date 2023/05/31 · openalex updated_date 2026/08/01

Abstract

Dynamic / ME-JEPA v2.0.0-rc1 is an audited, single-binary JEPA-style world-model runtime that runs three domains — counter_world, gridworld₅x5, and the new public-taxonomy career_taxonomy.v1 benchmark — on one RTX 5090 by TOML manifest alone. Every persisted operation flows through 25 typed RocksDB column families plus an append-only audit log; a separate verifier reads every claim back. The reproducibility artifact is a deterministically built Docker + Apptainer (SIF) image with full SHA-256 hash chains and a one-command verify-aggregate path. What's new in v2.0.0-rc1 (vs. v1.0.0-rc5) Meaning-compression layer around the original DynamicJEPA runtime: pairwise instrument readings, constellation/centroid records, G_tau guard verification, signal-yield accounting, and MC-ratio reporting. Pairwise mutual-information audit outputs for the career_taxonomy domain. Cross-domain transfer evidence between counter_world, gridworld₅x5, and career_taxonomy. Bundle-to-training-corpus exporter emitting verified (state, action, expected_persisted_delta, actual_readback, verifier_status, error_code, remediation) rows. Release-D source-of-truth gate: aggregate passed_count=5 / failed_count=0; reference check diff_count=0, manifest=passed, fsv=passed. Why this matters One binary, three domains, by TOML alone. A new domain ships as a strict TOML manifest; the Rust runtime carries no domain-specific code branches. Five-seed paper-small results on commodity hardware. Career-taxonomy validation cosine 0.905 ± 0.044 (BCa 95% CI 0.861–0.931); test cosine 0.868 ± 0.068 (BCa 95% CI 0.806–0.912); validation action-contrast accuracy 0.960 ± 0.043. The negative-action ablation A3 collapses to validation cosine 0.790 — exactly as the design predicts and the verifier confirms. Every cited number reads from a CSV in the verified release bundle; hand-typed numbers are forbidden. A reproduction reviewers can actually run. One command returns status: ok, diff_count: 0, manifest: passed, fsv: passed, passed_count: 5 byte-for-byte. Each evidence bundle is also a new class of training data. The artifact argues, with citations, that a bundle is a training-data class distinct from web text, pure model-generated synthetic data, and rejection-sampled chain-of-thought. The proposed name is audited execution traces with cryptographic provenance. Most of the bytes are deterministic functions of the public O*NET ZIP rather than model output; model outputs are paired in the same record with the deterministic observed target plus random-init / shuffled-target baselines; counterfactual ablations and failed runs are persisted as first-class records. Used carefully, this corpus carries substantially lower model-collapse hazard than typical synthetic data (Shumailov et al., Nature 2024; Gerstgrasser et al., 2024). Failed runs are kept on purpose. Most synthetic-data pipelines discard rejected samples; this artifact preserves them. That choice is the difference between recursion-with-replacement (which Shumailov showed causes collapse) and accumulation (which Gerstgrasser showed avoids it). Reviewer command CONTEXT_GRAPH_BIN=bin/context-graph ./reproduce.sh verify-aggregate --aggregate bundles/20260502T_release_dgate_v3_paper_small_multiseed_aggregate --reference reference_release_v2.0.0.json --json Archive contents and SHA-256 hashes mejepa₅090_artifact_v2.0.0_rc1.tar.gz — complete release archive, SHA-256 91461515e29c851017ef9bdd5fb584ed6bc404abedca52be4b8107fb3d4b1a01, size 7,430,206,664 bytes. bin/context-graph — single Rust binary that runs the entire pipeline (ingest, panel, train, predict, plan, verify), SHA-256 e6013ccb72e1878e6229e5e43c8d317b5db5c46ba794b99bdbbe91a942589494. container/contextgraph-mejepa.docker.tar — deterministic Docker image, SHA-256 830304d56887b54d72c34f51c05176f2c39557e97829468fab379f403ae1940b. container/contextgraph-mejepa.sif — Apptainer/SingularityCE image built from the Docker tar via docker-archive bootstrap, SHA-256 78c87c55406996d711f4ecff7c12fe66610dc8d53110766784cf7ed80f386bad. bundles/20260502T_release_dgate_v3_paper_small_multiseed_aggregate — five-seed aggregate with paper tables, FSV report, and pairwise-MI audit. bundles/20260502T_release_dgate_v3_paper_small_seed42 — one full passed seed bundle for manual source-of-truth inspection. reference_release_v2.0.0.json — checked-in numeric and structural release reference, SHA-256 70eebc518feb8786c006cbc02bd4df00bebaa42ac29fdc6998d60da74acd9b57. archive_file_hashes.json, release_manifest.json, checksums.txt — byte-level provenance. Headline numbers Career-taxonomy validation cosine 0.905 ± 0.044, BCa 95% CI 0.861–0.931 (n = 5 seeds, BCa bootstrap 10,000 iterations, seed 20260430). counter_world: 0.99963 / 0.99916 (480 trainable parameters). gridworld₅x5: 0.99473 / 0.99472 (16,960 trainable parameters). career_taxonomy A1 panel: 15,344 trainable parameters. Full FSV integration suite passes in 190.92 s. Full release-C gate (5 seeds + aggregate + container build + container verify + manual edges) fits in ~2 hours wall time on the documented host. Hardware and runtime NVIDIA GeForce RTX 5090 (sm₁20, 32 GiB VRAM, CUDA 13.2). Reproduction requires a comparable GPU and uses the NVIDIA container runtime with --gpus all. Apptainer / SingularityCE 4.x recommended for the SIF execution path; Docker 24+ for the Docker tar path. Data boundary The career benchmark is generated deterministically from the public O*NET 30.2 text database. It is a public taxonomy benchmark, not real recruiting history. The packaged selected bundle records O*NET source ZIP SHA-256 b5479271931796b838f7173dc0f673a9ec961b7833ac87168fd11e92e7453741 and row count 1000 in data_manifest.json. Where the paper is A two-column publication-ready LaTeX manuscript (17 pages) is included inside the archive at docs/5090jepa/researchpaper/paper.tex and paper.pdf. The paper develops the runtime, the public-taxonomy benchmark, the full-state-verification protocol, and the new training-data class argument. Citation Cite the specific version DOI for reproducibility: 10.5281/zenodo.19977981. To cite all versions of this artifact, use the concept DOI: 10.5281/zenodo.19953950. License Code: PolyForm Noncommercial 1.0.0. Bundle data and generated paper tables: CC BY 4.0. Third-party data (O*NET 30.2): retains its CC BY 4.0 license; full attribution in data_manifest.json and THIRD_PARTY_LICENSES/ONET.md inside every bundle. Reproducibility note Per PyTorch's reproducibility guidance, exact GPU bit-reproducibility across releases or hardware is not guaranteed; the release reference therefore uses a structural + numeric-tolerance schema rather than byte-equal model-weight diffs. Failed runs are intentionally preserved in the archive as evidence; the training-data class argument depends on this property.

Summary

The authors identify 'model collapse,' a degenerative process where generative models trained on the output of their predecessors forget the tails of the original data distribution. This effect is demonstrated across Gaussian Mixture Models, Variational Autoencoders, and Large Language Models, showing that access to human-generated data is essential to prevent this decay.

machine-generated · gemma4:31b

In simple words

Imagine you make a copy of a copy of a copy. Each time, some small details are lost. If a computer program learns from its own old work instead of from real people, it starts to forget the rare things that happen in the world. Eventually, it only remembers the most common things and then makes mistakes. It becomes like a blind person trying to describe a world they have only heard about from other blind people.

machine-generated · gemma4:31b

Outline

• Introduction — Introduces model collapse as a process where models forget the true data distribution when trained on synthetic data.
• Related work — Distinguishes model collapse from catastrophic forgetting and data poisoning.
• What is Model Collapse? — Defines early and late model collapse and identifies statistical and functional approximation errors as the causes.
• Theoretical intuition — Uses mathematical toy models (discrete distributions and 1D Gaussians) to show why divergence from the original distribution is inevitable with finite sampling.
• Evaluation — Empirically tests model collapse in GMMs, VAEs, and fine-tuned LLMs (OPT-125m).
• Discussion and Conclusion — Discusses the risks of automated data poisoning at scale and the importance of preserving human-generated data.

machine-generated · gemma4:31b

Argument

1. Generative models are trained on datasets that are approximations of a true distribution.
   Assumption
2. Finite sampling leads to statistical approximation errors, causing the model to miss rare events (the tails).
   Mathematical proof using discrete distributions and Markov Chains
3. Functional approximation errors further compound these deviations from the true distribution.
   Theoretical analysis of 1D Gaussians and Wasserstein-2 distance
4. When a model is trained on data generated by its predecessor, these errors accumulate over generations.
   Empirical results from GMMs and VAEs showing convergence to point estimates
5. This recursive loop leads to 'model collapse,' where the model eventually misperceives reality entirely.
   Experimental evidence using OPT-125m LLM on wikitext2 dataset

machine-generated · gemma4:31b

Assumptions

• The training process for subsequent generations involves sampling from the previous model's distribution. [stated]
• Models are trained on finite datasets, meaning they cannot perfectly capture the original distribution. [stated]
• The noise in functional approximation is orthogonal to the sample mean (Assumption 3). [stated]
• Future LLMs will inevitably be trained on web-scraped data containing significant amounts of model-generated content. [unstated]

machine-generated · gemma4:31b

Claims

• Model collapse is a universal phenomenon across different types of generative models (GMMs, VAEs, LLMs). [experiment]
• Training on model-generated data causes the loss of information about the tails of the original distribution. [experiment]
• To avoid model collapse, access to genuine human-generated content is essential. [experiment]
• The risk (Wasserstein distance) from the true distribution diverges linearly for a constant sample size across generations. [proof]

machine-generated · gemma4:31b

Methods

• n: not stated
• population: n/a
• study type: simulation

machine-generated · gemma4:31b

Limitations

admitted by authors:

• The authors note that training a moderately large language model from scratch is computationally expensive and has a high carbon footprint, so they used fine-tuning of pre-trained models (OPT-125m) as a proof-of-concept instead.
• They acknowledge that the results for LLMs are based on a specific dataset (wikitext2) and a specific model architecture.

noticed by the model, not admitted:

• The authors do not explicitly discuss how different sampling strategies beyond beam search might affect the rate of model collapse.
• The paper does not provide a quantitative threshold for the 'superlinear' increase in sample size required to prevent collapse across different model types.

machine-generated · gemma4:31b

Key equations

• Xnj = μ + (σ)/(√(M0))Z1 + … + σ√S1×…× Sn Znj — Describes the recursive accumulation of sampling errors across generations for a 1D Gaussian distribution.
• Var(Xnj) = σ2(1+(n)/(M)) — Shows that variance diverges linearly over generations when sample size M is constant, resembling a random walk.
• 𝔼[Rn+1W2] ≥ TrΣ(∑i=0n (1)/(Mi)) + ∑i=1n+1 𝔼[‖εi‖2] — Establishes a lower bound on the distance from the original distribution, proving that errors compound unless sample sizes increase superlinearly.

machine-generated · gemma4:31b

Proof sketch

1. The authors model recursive training as a Markov Chain where delta functions are absorbing states, ensuring eventual convergence to a single point.
2. They use a 1D Gaussian toy model to analytically show that variance diverges linearly over generations due to finite sampling noise.
3. They extend this to a general lower bound using the Wasserstein-2 distance, demonstrating that risk grows with each generation.
4. The argument concludes that without superlinear growth in sample size or access to original human data, generative models inevitably diverge from reality.

machine-generated · gemma4:31b

Open questions

• How can content generated by LLMs be tracked at scale to ensure data provenance?
  Without tracking, it is difficult to filter out synthetic data and prevent model collapse in future training sets.

machine-generated · gemma4:31b

Cited by

• Content Creation with Spillovers: An Incentive Design Approach
• Learning from Synthetic Data without Model Collapse in Iterative Instruction Tuning
• A theoretical basis for model collapse in recursive training
• Fence: Specialized SLM Guardrails for LLM Applications
• Distinguishing Artificial from Authentic: Evaluating LLMs for Detecting LLM-Generated Content
• Cosine capital: Large language models and the embedding of all things
• LLMs Can Get "Brain Rot": A Pilot Study on Twitter/X
• Anti-Regulatory AI: How "AI Safety" is Leveraged Against Regulatory Oversight
• To Improve Literacy, Improve Equality in Education, Not Large Language Models
• Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
• Governing artificial intelligence means governing data: (Re)setting the agenda for data justice
• Selecting Language Models for Social Science: Start Small, Start Open, and Validate
• Market Design for AI: Beyond the Copyright Binary
• Code2Doc: A Quality-First Curated Dataset for Code Documentation
• Few-Shot Learning of a Graph-Based Neural Network Model Without Backpropagation
• Sharing Knowledge without Sharing Data: Stitches can improve ensembles of disjointly trained models
• The Meta-Prompting Protocol: Orchestrating LLMs via Adversarial Feedback Loops
• Entropy-Reservoir Bregman Projection: An Information-Geometric Unification of Model Collapse
• Entropy Collapse: A Universal Failure Mode of Intelligent Systems
• On the Dangers of Bootstrapping Generation for Continual Learning and Beyond
• Semantic Reconstruction of Adversarial Plagiarism: A Context-Aware Framework for Detecting and Restoring "Tortured Phrases" in Scientific Literature
• ValuePilot: A Two-Phase Framework for Value-Driven Decision-Making
• Group Selection as a Safeguard Against AI Substitution
• The Necessity of Imperfection:Reversing Model Collapse via Simulating Cognitive Boundedness
• A Customer Journey in the Land of Oz: Leveraging the Wizard of Oz Technique to Model Emotions in Customer Service Interactions
• Data Value in the Age of Scaling: Understanding LLM Scaling Dynamics Under Real-Synthetic Data Mixtures
• The Alignment Game: A Theory of Long-Horizon Alignment Through Recursive Curation
• Stabilizing Self-Consuming Diffusion Models with Latent Space Filtering
• Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk
• Forgetting is Everywhere
• Generate, Evaluate, Iterate: Synthetic Data for Human-in-the-Loop Refinement of LLM Judges
• Why Less is More (Sometimes): A Theory of Data Curation
• A social path to human-like artificial intelligence
• (A)I Cannot See Them: A Situated Reflection on the Simulation of Historical Figures
• Reclaiming AI as a theoretical tool for cognitive science
• The seven roles of generative AI: Potential & pitfalls in combatting misinformation
• VERITAS: Leveraging Vision Priors and Expert Fusion to Improve Multimodal Data
• KORMo: Korean Open Reasoning Model for Everyone
• Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
• Characterizing Model Behavior Under Synthetic Data Training: An Empirical Study Across Scales and Mixing Ratios
• Steering an Active Learning Workflow Towards Novel Materials Discovery via Queue Prioritization
• Learning in an Echo Chamber: Online Learning with Replay Adversary
• CLPO: Curriculum Learning meets Policy Optimization for LLM Reasoning
• AI and memory
• Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression
• Lessons from complex systems science for AI governance
• Adversarial machine learning :
• Synthetic data as meaningful data. On Responsibility in data ecosystems
• Rethinking the Role of Text Complexity in Language Model Pretraining
• ForTIFAI: Fending Off Recursive Training Induced Failure for AI Model Collapse
• The Anti-Ouroboros Effect: Emergent Resilience in Large Language Models from Recursive Selective Feedback
• Abex-rat: Synergizing Abstractive Augmentation and Adversarial Training for Classification of Occupational Accident Reports
• No Clustering, No Routing: How Transformers Actually Process Rare Tokens
• Making AI Inevitable: Historical Perspective and the Problems of Predicting Long-Term Technological Change
• CAMF: Collaborative Adversarial Multi-agent Framework for Machine Generated Text Detection
• BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining
• Approaching the integration of large language models in the parliamentary workspace
• Machine culture
• Modeling Annotator Disagreement with Demographic-Aware Experts and Synthetic Perspectives
• Model Misalignment and Language Change: Traces of AI-Associated Language in Unscripted Spoken English
• Flow Matching Policy Gradients
• Linguistic and Embedding-Based Profiling of Texts generated by Humans and Large Language Models
• Synthetic Voice Data for Automatic Speech Recognition in African Languages
• Using Sign Language Production as Data Augmentation to enhance Sign Language Translation
• Model Collapse Is Not a Bug but a Feature in Machine Unlearning for LLMs
• Losing our Tail, Again: (Un)Natural Selection & Multilingual LLMs
• SAILViT: Towards Robust and Generalizable Visual Backbones for MLLMs via Gradual Feature Refinement
• Multi-interaction TTS toward professional recording reproduction
• FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale
• Enhancing Reasoning Capabilities in SLMs with Reward Guided Dataset Distillation
• What Matters in LLM-generated Data: Diversity and Its Effect on Model Fine-Tuning
• Unnatural Images: On AI-Generated Photographs
• LLMs in Coding and their Impact on the Commercial Software Engineering Landscape
• Recycling the Web: A Method to Enhance Pre-training Data Quality and Quantity for Language Models
• We are an input, now. Generative AI and the death of the journalist as human memory-maker
• Gaussian mixture models as a proxy for interacting language models
• ContextQFormer: A New Context Modeling Method for Multi-Turn Multi-Modal Conversations
• Beyond Monoliths: Expert Orchestration for More Capable, Democratic, and Safe Language Models
• When Your Own Output Becomes Your Training Data: Noise-to-Meaning Loops and a Formal RSI Trigger
• Can Large Reasoning Models Self-Train?
• LLM Web Dynamics: Tracing Model Collapse in a Network of LLMs
• Bayesian Meta-Reinforcement Learning with Laplace Variational Recurrent Networks
• Diverse, not Short: A Length-Controlled Data Selection Strategy for Improving Response Diversity of Language Models
• Reasoning in Neurosymbolic AI
• Collective Translation as Forking (分岔)
• Artificial intelligence and illusions of understanding in scientific research
• AutoFOAM: The Self-Refining Autonomous OpenFOAM Agent
• LLM Watermarking Using Mixtures and Statistical-to-Computational Gaps
• Long-Tail Knowledge in Large Language Models: Taxonomy, Mechanisms, Interventions and Implications
• Generalistic or Specific Embeddings, Which is Better? An Empirical Study on Search for Clinical Coding in Non-English Languages
• On the Limits of Self-Improving in Large Language Models: The Singularity Is Not Near Without Symbolic Model Synthesis
• From Collapse to Improvement: Statistical Perspectives on the Evolutionary Dynamics of Iterative Training on Contaminated Sources
• AI Survival Stories: a Taxonomic Analysis of AI Existential Risk
• The Fairness Collapse Phenomenon: Bias Amplification in Language Models Trained on Synthetic Data
• Feeding LLM Annotations to BERT Classifiers at Your Own Risk
• Watermarking Needs Input Repetition Masking
• OpenTuringBench: An Open-Model-based Benchmark and Framework for Machine-Generated Text Detection and Attribution
• Leveraging Robust Optimization for LLM Alignment under Distribution Shifts
• Recursive Training Loops in LLMs: How training data properties modulate distribution shift in generated data?
• Model collapse [wikipedia]

Discussions

• The Curse of Recursion: Training on Generated Data Makes Models Forget [hn, 170 points, 117 comments]
• The Curse of Recursion: Training on generated data makes models forget (2023) [hn, 122 points, 107 comments]
• AI Will Eat Itself? This AI Paper Introduces A Phenomenon Called Model Collapse That Refers To A Degenerative Learning Process Where Models Start Forgetting Improbable Events Over Time. (27.05.2023 p) [lemmy, 16 points, 4 comments]
• ソース arxiv.org/abs/2305.17493 日本語まとめ news.yahoo.co.jp/expert/artic... [bsky, 16 points, 0 comments]
• The term for the consequence is "model collapse" [bsky, 12 points, 0 comments]
• the same people published this on arXiv over a year ago. I'm glad they got a Nature paper out of it, but this has been a thing that's been known since about a month after ChatGPT dropped. arxiv.org/a [bsky, 10 points, 1 comments]
• Pesquisa recente chegando à conclusão que “AI trained on AI-generated data gets dumber every generation until it forgets what real human data looked like.” O collapso do modelo atual é iminente. Quem [bsky, 10 points, 1 comments]
• Common misconception: "Model collapse" and synthetic data [lemmy, 9 points, 10 comments]
• model collapse: a learning process where models forget improbable events and converge to a single estimate tl;dr - we still need human-generated content to maintain the benefits of large-scale data t [bsky, 8 points, 0 comments]
• yeah it's worse than that: https://arxiv.org/abs/2305.17493v2 [bsky, 8 points, 2 comments]
• interesting: "We find that use of model-generated content in training causes irreversible defects in the resulting models, where tails of the original content distribution disappear." [bsky, 7 points, 1 comments]
• Training AI on other AI causes models to collapse (original title : The AI is eating itself) [lemmy, 7 points, 3 comments]
• Har verkligen inte järnkoll på utvecklingen av AI, men pga jobbat med prediktiv modellering under en stor del av mitt yrkesliv är jag exakt noll förvånad. Skit in skit ut 🤷‍♀️ [bsky, 6 points, 1 comments]
• The Curse of Recursion: Training on Generated Data Makes Models Forget [hn, 5 points, 0 comments]
• THE CURSE OF RECURSION: TRAINING ON GENERATED DATA MAKES MODELS FORGET [lemmy, 5 points, 0 comments]
• I’ve conjectured this for years, but seeing Papernot and Shumailov on the paper makes me feel really confident in the findings: https://arxiv.org/abs/2305.17493 Existential risk 🙄🙄🙄🙄 [bsky, 4 points, 0 comments]
• LLMs will ALWAYS rely on human labor. This is what happens when you train one on training data generated by another LLM after a few generations. https://arxiv.org/pdf/2305.17493v2.pdf [bsky, 4 points, 1 comments]
• achei um aqui que parece interessante, depois vou ler. arxiv.org/abs/2305.17493 [bsky, 4 points, 3 comments]
• Here is an example of the research I was thinking of when I wrote "takes an average of an average of an average" https://arxiv.org/abs/2305.17493v2 [bsky, 2 points, 0 comments]
• This is indeed a problem - some more detailed analysis of it can be found here: arxiv.org/abs/2305.17493 [bsky, 2 points, 1 comments]
• As the internet fills up with AI generated text, new LLM AI models trained on these texts created by their precursors get progressively narrower, more generic, and less valuable - "model collapse". PD [bsky, 2 points, 1 comments]
• tiré de l'article "The Curse of Recursion" arxiv.org/pdf/2305.174... [bsky, 2 points, 1 comments]
• There's also a paper on this effect: arxiv.org/abs/2305.17493 In summary: models train to estimate a probably distribution. Output loses outliers. Repeat until mode collapse. Part of the reason I find [bsky, 2 points, 0 comments]
• vgl der fluch der rekursion arxiv.org/abs/2305.174... [bsky, 2 points, 1 comments]
• "What will happen to GPT-{n} once LLMs contribute much of the language found online? We find that use of model-generated content in training causes irreversible defects in the resulting models, where [bsky, 2 points, 0 comments]
• And LLMs increasingly fed with LLM generated content might become (even more) useless much faster than one might anticipate: arxiv.org/pdf/2305.17493 [bsky, 2 points, 1 comments]
• was linked this study the other day, was a good read arxiv.org/abs/2305.174... [bsky, 1 points, 0 comments]
• Since the current generation of LLMs seem to need meaningful, human-generated content as training data it seems possible that the demand for it might actually rise in some contexts https://arxiv.org/a [bsky, 1 points, 1 comments]
• Turns out that the AI Ouroborous becomes a centered dot! https://arxiv.org/abs/2305.17493v2 [bsky, 1 points, 0 comments]
• arxiv.org/pdf/2305.17493 "THE CURSE OF RECURSION: TRAINING ON GENERATED DATA MAKES MODELS FORGET" in other words: LLMs get worse over time because they start to incorporate their own generated texts. [bsky, 1 points, 1 comments]
• The system eats itself: AI trained on AI-generated content, produced by models that were themselves trained on AI-generated content, until the original signal of real human data is lost entirely arxiv [bsky, 1 points, 0 comments]
• Not sure how I missed reading this paper last year. Remediation in progress.... arxiv.org/abs/2305.17493 [bsky, 1 points, 0 comments]
• "@[email protected] The use of model-generated content in training causes irreversible defects in the resulting models, where tails of the original content distribution disappear. The Curse of Recu [bsky, 1 points, 0 comments]
• The paper in question 🤖 arxiv.org/abs/2305.17493 [bsky, 1 points, 0 comments]
• Generative models making and then suffering from their own space junk problem: [bsky, 1 points, 0 comments]
• This is real. It's being called "model collapse," and the deterioration happens a LOT faster than most people think. [bsky, 1 points, 0 comments]
• arxiv.org/abs/2305.17493 [bsky, 1 points, 0 comments]
• Recursion is going to be even worse as models run out of data and start ingesting other AI generated content. arxiv.org/pdf/2305.17493 [bsky, 1 points, 0 comments]
• @wikisteff.bsky.social Recursive enshittification. I'd say it's approaching 90%. arxiv.org/pdf/2305.17493 [bsky, 1 points, 0 comments]
• Gore Verbinski tenía razón. arxiv.org/pdf/2305.17493 [bsky, 0 points, 0 comments]
• Hier gibt's die Studie arxiv.org/abs/2305.17493 [bsky, 0 points, 0 comments]
• LLM inbreeding? Nothing really good comes from it. #SALAMI [bsky, 0 points, 0 comments]
• Alla fine tutto sarà una fotocopie sbiadita di quella che era creatività, ingegno e fantasia. Tutto bruciato insieme alle foreste. arxiv.org/pdf/2305.17493 [bsky, 0 points, 0 comments]
• The Curse of Recursion - doch da die Kuratierung, also die Auswahl und Aufbereitung von Trainingsdaten auch für LLM bereits heute ein wichtiges Thema ist - LLM suchen sich nicht einfach fortwährend di [bsky, 0 points, 0 comments]
• via Arthur Charpentier ⏚: "The Curse of Recursion: Training on Generated Data Makes Models Forget" https://arxiv.org/abs/2305.17493 "In this paper we consider what the future might hold. What will hap [bsky, 0 points, 1 comments]
• I would be interested in how the targeted use and limitation to "one generation of reuse" interacts with the study on model-colapse in the case of iterative and continuous feeding of one models output [bsky, 0 points, 1 comments]
• This paper by Ilia Shumailov et al is super fascinating - they show that LLMs degenerate if they're trained on data generated by LLMs. [bsky, 0 points, 0 comments]
• Er kwam aardig wat kritiek op dit stuk, maar de schrijvers lijken het toch goed gezien te hebben arxiv.org/pdf/2305.17493 [bsky, 0 points, 0 comments]
• W8 WHAT?! Altman of all people unaware of degradation of distribution after training on AI output? Or if he just generating hype credibility be damned? I need emotional support cookies. arxiv.org/a [bsky, 0 points, 0 comments]
• arxiv.org/pdf/2305.17493 The curse of recursion [bsky, 0 points, 0 comments]
• "Model collapse." http://arxiv.org/abs/2305.17493 Imagine an AI trained on the crap that comes out of ChatGPT and then imagine ten generations of that. [bsky, 0 points, 0 comments]
• https://arxiv.org/abs/2305.17493v2) [bsky, 0 points, 1 comments]
• The Curse of Recursion: Training on generated data makes models forget (2023) https://arxiv.org/abs/2305.17493 [comments] [39 points] [bsky, 0 points, 0 comments]
• https://bsky.app/profile/news.ycombinator.com.web.brid.gy/post/3lcpo6utrp5f2 [bsky, 0 points, 0 comments]
• The Curse of Recursion: Training on generated data makes models forget (2023) (arxiv.org) Main Link | Discussion [bsky, 0 points, 0 comments]
• Interesting research into #AI model collapse: "The value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of content generated by LLMs in d [bsky, 0 points, 0 comments]
• The Curse of Recursion: Training on generated data makes models forget (2023) https://arxiv.org/abs/2305.17493 https://news.ycombinator.com/item?id=42286395 [bsky, 0 points, 0 comments]
• There's this one, too: arxiv.org/abs/2305.174... [bsky, 0 points, 0 comments]
• https://arxiv.org/abs/2305.17493 [bsky, 0 points, 0 comments]

Related

• AI models collapse when trained on recursively generated data
• Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
• Experimental evidence of progressive ChatGPT models self-convergence
• Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loop
• Recursive Training Loops in LLMs: How training data properties modulate distribution shift in generated data?