The wall confronting large language models

2025/07/25 by Peter V. Coveney, Sauro Succi, Coveney, Peter V. +1 · 18 voices · 4 citations
#cs.AI

paper · pdf · doi:10.48550/arxiv.2507.19703

Abstract

We show that the scaling laws which determine the performance of large language models (LLMs) severely limit their ability to improve the uncertainty of their predictions. As a result, raising their reliability to meet the standards of scientific inquiry is intractable by any reasonable measure. We argue that the very mechanism which fuels much of the learning power of LLMs, namely the ability to generate non-Gaussian output distributions from Gaussian input ones, might well be at the roots of their propensity to produce error pileup, ensuing information catastrophes and degenerative AI behaviour. This tension between learning and accuracy is a likely candidate mechanism underlying the observed low values of the scaling components. It is substantially compounded by the deluge of spurious correlations pointed out by Calude and Longo which rapidly increase in any data set merely as a function of its size, regardless of its nature. The fact that a degenerative AI pathway is a very probable feature of the LLM landscape does not mean that it must inevitably arise in all future AI research. Its avoidance, which we also discuss in this paper, necessitates putting a much higher premium on insight and understanding of the structural characteristics of the problems being investigated.

Citations

• Potemkin Understanding in Large Language Models
• Overtrained Language Models Are Harder to Fine-Tune
• AI models collapse when trained on recursively generated data
• AlphaFold predictions are valuable hypotheses and accelerate but do not replace experimental structure determination
• Training Compute-Optimal Large Language Models
• Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Attention Is All You Need

Cited by

• AI Needs Physics More Than Physics Needs AI
• On the Computability of Artificial General Intelligence
• Stop Saying "AI"
• The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems

Discussions

• While we've all been obsessing over this benchmark and that benchmark for the latest spiffy model, physics has been exploring the limits of Large Language Models. For applications where accuracy matte [bsky, 8 points, 0 comments]
• "That LLMs produce non-Gaussian output distributions from Gaussian inputs is the very mechanism that prevents LLMs from ever meeting the standards required of scientific inquiry." arxiv.org/abs/2507.1 [bsky, 6 points, 0 comments]
• The wall confronting large language models [lobsters, 6 points, 1 comments]
• The wall confronting large language models [hn, 5 points, 1 comments]
• Or if you'd like to read the paper he's discussing. arxiv.org/abs/2507.19703 [bsky, 3 points, 0 comments]
• This has been my feeling for a couple years: LLMs are at or near their ceiling for capabilities, and will not improve dramatically from here, for reasons technical, practical, legal and economic. Rece [bsky, 1 points, 1 comments]
• The wall confronting large language models https://arxiv.org/abs/2507.19703 [bsky, 0 points, 0 comments]
• The wall confronting large language models https://arxiv.org/abs/2507.19703 (https://news.ycombinator.com/item?id=45114579) [bsky, 0 points, 0 comments]
• The wall confronting large language models #llms #generativeai #limits [bsky, 0 points, 0 comments]
• The wall confronting large language models https://lobste.rs/s/umsj7d #scaling #ai [bsky, 0 points, 0 comments]
• Hitting those scaling limits is a predictable hurdle. Interesting to see the specific bottlenecks researchers are identifying in LLMs. A key challenge to address. 🤖 #ai The wall confronting large lan [bsky, 0 points, 0 comments]
• The wall confronting large language models arxiv.org/abs/2507.19703 [bsky, 0 points, 0 comments]
• https://arxiv.org/abs/2507.19703 大規模言語モデル(LLM)の性能を決定するスケーリング則について解説されています。 LLMが予測の不確実性を改善する能力には限界があることを示しています。 学習能力と精度との間の緊張関係が、スケーリング成分の低い値の根本原因であると主張しています。 [bsky, 0 points, 0 comments]
• www.arxiv.org/pdf/2507.19703 [bsky, 0 points, 0 comments]
• The wall confronting large language models https://arxiv.org/abs/2507.19703 (https://news.ycombinator.com/item?id=45114579) [bsky, 0 points, 0 comments]
• "[R]aising their reliability to meet the standards of scientific inquiry is intractable by any reasonable measure ... [T]he very mechanism which fuels much of the learning power of LLMs ...might well [bsky, 0 points, 0 comments]
• The wall confronting large language models [bsky, 0 points, 0 comments]
• The wall confronting large language models https://arxiv.org/abs/2507.19703 https://news.ycombinator.com/item?id=45114579 [bsky, 0 points, 0 comments]

Related

• Do Large Language Models Know What They Are Capable Of?
• Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve
• Information-Theoretic Limits of Reliability and Scaling in Language Models
• On the Fundamental Limits of LLMs at Scale
• Large Language Models Must Be Taught to Know What They Don't Know