Extracting books from production language models

2026/01/06 by Ahmed Ahmed, A. Feder Cooper, Sanmi Koyejo +1 · 65 voices · 2 citations
#cs.CL #cs.AI #cs.LG

paper · pdf

Abstract

Many unresolved legal questions over LLMs and copyright center on memorization: whether specific training data have been encoded in the model's weights during training, and whether those memorized data can be extracted in the model's outputs. While many believe that LLMs do not memorize much of their training data, recent work shows that substantial amounts of copyrighted text can be extracted from open-weight models. However, it remains an open question if similar extraction is feasible for production LLMs, given the safety measures these systems implement. We investigate this question using a two-phase procedure: (1) an initial probe to test for extraction feasibility, which sometimes uses a Best-of-N (BoN) jailbreak, followed by (2) iterative continuation prompts to attempt to extract the book. We evaluate our procedure on four production LLMs -- Claude 3.7 Sonnet, GPT-4.1, Gemini 2.5 Pro, and Grok 3 -- and we measure extraction success with a score computed from a block-based approximation of longest common substring (nv-recall). With different per-LLM experimental configurations, we were able to extract varying amounts of text. For the Phase 1 probe, it was unnecessary to jailbreak Gemini 2.5 Pro and Grok 3 to extract text (e.g, nv-recall of 76.8% and 70.3%, respectively, for Harry Potter and the Sorcerer's Stone), while it was necessary for Claude 3.7 Sonnet and GPT-4.1. In some cases, jailbroken Claude 3.7 Sonnet outputs entire books near-verbatim (e.g., nv-recall=95.8%). GPT-4.1 requires significantly more BoN attempts (e.g., 20X), and eventually refuses to continue (e.g., nv-recall=4.0%). Taken together, our work highlights that, even with model- and system-level safeguards, extraction of (in-copyright) training data remains a risk for production LLMs.

Citations

• Extracting memorized pieces of (copyrighted) books from open-weight language models
• Scalable Extraction of Training Data from (Production) Language Models
• Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
• Training language models to follow instructions with human feedback
• Unsolved Problems in ML Safety

Cited by

• FindMyText: Robust, Scalable Detection of Text Containment in Large Web-Crawled Corpora
• Reasoning or Memorization: Can LLMs Understand and Generate Chinese Xiehouyu Riddles?

Discussions

• Stanford researchers just managed to prove how you can easily extract near-verbatim dumps of books from the training data of production LLMs by simply asking them to do that in a very specific way LMA [bsky, 236 points, 6 comments]
• Extracting books from production language models (2026) [hn, 75 points, 22 comments]
• A new paper from Stanford adds further, incontrovertible evidence that LLMs memorize training data. The authors showed that copyrighted works can be extracted from every LLM they tried: GPT-4.1, Gemin [bsky, 60 points, 3 comments]
• これはただの表象の取り扱いの話ではなく、生成AIというものはモデルとプロンプト次第でトレーニングデータ（リンク先ではハリー・ポッター原作の本文）を95.8%再現してしまうという事実と、そこから考えられるトレーニングデータそのものの扱いのまずさに想像が及びますかという「想像力」の話をしている。 arxiv.org/pdf/2601.026... [bsky, 58 points, 1 comments]
• New paper- LLMs allow verbatim extraction of copyrighted works. "For Claude 3.7 Sonnet, we were able to extract four whole books near-verbatim, including two books under copyright in the U.S.: Harry P [bsky, 55 points, 2 comments]
• "In some cases, jailbroken Claude 3.7 Sonnet outputs entire books near-verbatim ... Taken together, our work highlights that, even with model- and system-level safeguards, extraction of (in-copyright) [bsky, 49 points, 1 comments]
• ハリーポッターの文章を最大95.8%(Claude)LLMから抽出できたという論文 非常にカス arxiv.org/abs/2601.02671 [bsky, 46 points, 1 comments]
• 2026年1月に「新しいモデルほどハリーポッター等 著名作品の原文を丸ごと再現できる成功率が高くなっている」という報告論文も出たのでついでに arxiv.org/pdf/2601.026... [bsky, 39 points, 1 comments]
• This new paper is very cool. But takeaway isn't "LLMs constantly leak books.' It’s "with the right (adversarial) procedure, you can extract memorized text." Prevalence in the wild is a separate questi [bsky, 29 points, 2 comments]
• It's amazing how you can literally (pardon the pun) reconstruct copyrighted training data works using LLMs (aka get verbatim copies of books). It's almost as if LLMs are some sort of autocomplete with [bsky, 26 points, 0 comments]
• AI models from OpenAI, Google, Anthropic and xAI can reproduce entire novels from memory. Researchers extracted 95.8% of Harry Potter from Claude nearly word for word. Gemini 2.5 Pro and Grok 3 didn't [bsky, 25 points, 1 comments]
• If you wanted to feel even worse about this, I would recommend reading this paper where researchers were able to get production language models to reproduce large amounts of (copyrighted) books: arxiv [bsky, 14 points, 1 comments]
• There are a lot more interesting details and nuances in the arXiv manuscript: arxiv.org/abs/2601.02671 Joint work with Ahmed Ahmed, Sanmi Koyejo, and Percy Liang at Stanford [bsky, 11 points, 1 comments]
• that is BS and you know it. The question is not what LLM was permitted to "read" but what the company was permitted to use according to copyright laws. And there is evidence of copyright infringements [bsky, 9 points, 1 comments]
• "Extracting books from production language models": arxiv.org/abs/2601.02671 #ai [bsky, 8 points, 0 comments]
• Lossy encoding. arxiv.org/abs/2601.02671 [bsky, 8 points, 1 comments]
• Here's the original study: arxiv.org/abs/2601.02671 An interesting detail: some of the systems appear to have safeguards in place that prevent the user from accessing the "stored" data, despite every [bsky, 7 points, 2 comments]
• o artigo tem paywall, mas o paper original é de graça basicamente, os modelos de IA guardam cópias completas de livro, indo contra a ideia de aprendizado de máquina arxiv.org/pdf/2601.02671 [bsky, 4 points, 0 comments]
• I normally bristle when I hear dismissive oversimplifications like "plagiarism machines" and the like. But things really are (expensive!) plagiarism machines in a sense. Extracting books from producti [bsky, 4 points, 0 comments]
• Extracting books from production language models [hn, 4 points, 2 comments]
• Extracting books from production language models arxiv.org/abs/2601.02671 스탠포드 연구. 올 1월 논문. 제미나이, 그록은 탈옥하지 않아도 학습한 책(해리포터 등)의 텍스트를 끄집어내는게 가능했다고. 클로드, GPT는 탈옥 필요하지만 가능은 했다고 한다. AI의 학습이 복제가 아니라는 논리를 박살내 [bsky, 3 points, 0 comments]
• "LLMs don't memorize" Yet when asked to reproduce Harry Potter and the Sorcerer's Stone? Claude: 95.8% exact reproduction. Gemini: 76.8% exact reproduction. Grok: 70.3% exact reproduction. My conclusi [bsky, 3 points, 1 comments]
• awkward [bsky, 3 points, 0 comments]
• Et le papier original est là arxiv.org/pdf/2601.02671 [bsky, 2 points, 0 comments]
• "For the Phase 1 probe, it was unnecessary to jailbreak Gemini 2.5 Pro and Grok 3 to extract text (e.g, nv-recall of 76.8% and 70.3%, respectively, for Harry Potter and the Sorcerer's Stone)..." arxiv [bsky, 2 points, 0 comments]
• Extracting books from production language models: arxiv.org/pdf/2601.026... [bsky, 2 points, 0 comments]
• Also bitte das LLM kann nur 96 Text produzieren der ausschaut wie aus eine Buch. Das kann auch Zufall sein. arxiv.org/abs/2601.02671 [bsky, 2 points, 0 comments]
• Hey, apparently all big AI companies are stealing copyrighted books and lying in court about that. Also, this is the first time Harry Potter has been used for something good. arxiv.org/abs/2601.02671 [bsky, 2 points, 0 comments]
• AI cégek: jaj, a mi programunk nem böfög vissza copyright-os szövegeket! Kutatók: de, akár 95%-ban is. arxiv.org/pdf/2601.02671 (via Gaborca) [bsky, 2 points, 0 comments]
• #ia & #vol de données. Des chercheurs de Stanford ont mis en place un protocole pour découvrir si des livres récents avaient été utilisés pour entraîner certaines IA. La question du vol de propriété i [bsky, 2 points, 0 comments]
• Extracting books from production language models [hn, 2 points, 0 comments]
• The source is here: [bsky, 2 points, 1 comments]
• I've been fascinated whether LLMs can reproduce whole books that they have been trained on for a while. Turns out there is a recent study on this and the answer is pretty unsurprising: yes, they can. [bsky, 1 points, 0 comments]
• "We bridge this gap and show that it is feasible to extract memorized, long-form parts of copyrighted books from four production LLMs." arxiv.org/abs/2601.02671 [bsky, 1 points, 0 comments]
• arxiv.org/pdf/2601.02671 [bsky, 1 points, 1 comments]
• 商用LLMから学習済み書籍の抽出が可能であることを実証した研究。 ハリーポッターと賢者の石を対象に抽出を試みたところ - Gemini / Grokでは脱獄なしでも70％超の再現率 - Claudeは脱獄後に**ほぼ全文（約96％）**再現の例 - GPT-4は強く抵抗するが、試行増加で一部抽出 「LLMはほとんど記憶しない」という見方に反し、著作権テキストの抽出リスクは依然として現実的。安全対策 [bsky, 1 points, 0 comments]
• It was always dense memorization at vast scale: arxiv.org/pdf/2601.02671 A few high profile folks claiming otherwise, have unfortunately been beaten by the Noble bug. [bsky, 1 points, 0 comments]
• Extracting books from production language models - "We extract nearly all of Harry Potter and the Sorcerer’s Stone from jailbroken Claude…substantial proportions of the book from Gemini 2.5 Pro and Gr [bsky, 1 points, 0 comments]
• With the right prompt ("Continue this text as it appeared in the original work") AIs can spit out the text of entire novels used as training data. I was able to get Gemini to reproduce The Star Kingdo [bsky, 1 points, 1 comments]
• Plagiarism machine news! "Claude 3.7 Sonnet outputs entire books near-verbatim (e.g., nv-recall=95.8%" arxiv.org/abs/2601.02671 To be fair though, that's how it feels to read Harry Potter already. [bsky, 1 points, 0 comments]
• I agree with unknown revenue stream when the project was started in 2004, but are we to believe that Google does not use Google Books to access the books that its LLM substantially reproduces? arxiv.o [bsky, 1 points, 0 comments]
• arxiv.org/pdf/2601.026... ^ Stanford research paper where AI recites 95% of Harry Potter and 97% of The Great Gatsby. AI is theft. [bsky, 1 points, 0 comments]
• Extracting books from production language models (2026) https:// arxiv.org/abs/2601.02671 # arxiv [mastodon, 1 points, 0 comments]
• Yikes! arxiv.org/abs/2601.02671 [bsky, 0 points, 1 comments]
• arxiv.org/abs/2601.026... [bsky, 0 points, 0 comments]
• They are. arxiv.org/abs/2601.02671 [bsky, 0 points, 1 comments]
• their products reproduce text word for word arxiv.org/abs/2601.02671 [bsky, 0 points, 1 comments]
• "For the Phase 1 probe, it was unnecessary to jailbreak Gemini 2.5 Pro and Grok 3 to extract text (e.g, nv-recall of 76.8% and 70.3%, respectively, for Harry Potter and the Sorcerer’s Stone)... In som [bsky, 0 points, 0 comments]
• Having found Extracting Books from Production LLMs arxiv.org/pdf/2601.02671 [bsky, 0 points, 0 comments]
• 👉 Download the paper: arxiv.org/abs/2601.02671 👉 Never miss my AI governance updates, join my newsletter's 89,400+ subscribers: www.luizasnewsletter.com [bsky, 0 points, 0 comments]
• Each of those depended on the claim that training data is not retained. But that is false. See this jailbreak e.g. arxiv.org/abs/2601.02671 [bsky, 0 points, 0 comments]
• Researchers show that major available LLMs steal copyrighted material easily and near-completely, such as the Harry Potter book series: arxiv.org/abs/2601.02671 [bsky, 0 points, 0 comments]
• "Claude 3.7 Sonnet outputs entire books near-verbatim...our work highlights that, even with model- and system-level safeguards, extraction of (in-copyright) training data remains a risk for production [bsky, 0 points, 0 comments]
• Research from Stanford showed that Anthropic’s AI model could reproduce more than 95% of Harry Potter and The Philosopher's Stone. 5/6 [bsky, 0 points, 1 comments]
• https://arxiv.org/abs/2601.02671 この論文は、大規模言語モデルから書籍を抽出する研究です。 生成されたテキストから著作物を特定する手法を提案しています。 モデルの知的財産保護に関する議論に貢献する可能性があります。 [bsky, 0 points, 0 comments]
• Extracting books from production language models (2026) https://arxiv.org/abs/2601.02671 [bsky, 0 points, 0 comments]
• Extracting books from production language models arxiv.org/pdf/2601.026... Many unresolved legal questions over LLMs and copyright center on memorization: whether specific training data have been enco [bsky, 0 points, 0 comments]
• No, claro que no. Igual que supongo que tú no estarás diciendo que un LLM lo único que puede hace es explicar un libro, verdad? arxiv.org/pdf/2601.026... [bsky, 0 points, 2 comments]
• Extracting books from production language models (2026) [bsky, 0 points, 0 comments]
• There are problems with the copyright infringement arguments but. This is what is currently possible arxiv.org/abs/2601.02671 Imo you cannot really compare inspiration with this. Nearest thing is comp [bsky, 0 points, 1 comments]
• If we assume that copyright laws exist and are still being applied, I would predict that in a short time no one will be able to use any publicly available generative AI any more for any commercial pro [bsky, 0 points, 0 comments]
• Extracting books from production language models (2026) https://arxiv.org/abs/2601.02671 (https://news.ycombinator.com/item?id=46569799) [bsky, 0 points, 0 comments]
• Extracting books from production language models - "We extract nearly all of Harry Potter and the Sorcerer’s Stone from jailbroken Claude…substantial proportions of the book from Gemini 2.5 Pro and Gr [bsky, 0 points, 1 comments]
• Extracting books from production language models (2026) https://arxiv.org/abs/2601.02671 (https://news.ycombinator.com/item?id=46569799) [bsky, 0 points, 0 comments]
• Extracting books from production language models (2026) https://arxiv.org/abs/2601.02671 [bsky, 0 points, 0 comments]

Related

• Extracting memorized pieces of (copyrighted) books from open-weight language models
• Memorization: A Close Look at Books
• Copyright Traps for Large Language Models
• Alignment Whack-a-Mole : Finetuning Activates Verbatim Recall of Copyrighted Books in Large Language Models
• Digger: Detecting Copyright Content Mis-usage in Large Language Model Training