Readers Prefer Outputs of AI Trained on Copyrighted Books over Expert Human Writers

2025/10/15 by Tuhin Chakrabarty, Jane C. Ginsburg, Chakrabarty, Tuhin +3 · 6 voices
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computers and Society (cs.CY) #FOS: Computer and information sciences #cs.AI #cs.CL #cs.CY

paper · pdf · doi:10.48550/arxiv.2510.13939

arxiv published 2025/10/15 · arxiv updated 2026/03/17

Abstract

The use of copyrighted books for training AI has sparked lawsuits from authors concerned about AI generating derivative content. Yet whether these models can produce high-quality literary text emulating authors' voices remains unclear. We conducted a preregistered study comparing MFA-trained writers with three frontier models (ChatGPT, Claude, Gemini) writing up to 450-word excerpts emulating 50 award-winning authors' styles. In blind pairwise evaluations by 28 MFA-trained readers and 516 college-educated general readers, AI text from in-context prompting was strongly disfavored by MFA readers for stylistic fidelity (OR=0.16) and quality (OR=0.13), while general readers showed no fidelity preference (OR=1.06) but favored AI for quality (OR=1.82). Fine-tuning ChatGPT on authors' complete works reversed these results: MFA readers favored AI for fidelity (OR=8.16) and quality (OR=1.87), with general readers showing even stronger preference (fidelity OR=16.65; quality OR=5.42). Both groups preferred fine-tuned AI, but the writer-type X reader-type interaction remained significant (p=0.021 for fidelity; p<10^-4 for quality), indicating general readers favored AI by a wider margin. Effects are robust under cluster-robust inference and generalize across authors in heterogeneity analyses. Fine-tuned outputs were rarely flagged as AI-generated (3% vs. 97% for prompting) by leading detectors. Mediation analysis shows fine-tuning eliminates detectable AI quirks that penalize in-context outputs, altering the nexus between detectability and preference. While not accounting for effort to transform AI output into publishable prose, the median fine-tuning cost of 81 per author represents a 99.7% reduction versus typical writer compensation. Author-specific fine-tuning enables non-verbatim AI writing preferred over expert human writing, providing evidence relevant to copyright's fourth fair-use factor.

Citations

• ParaPO: Aligning Language Models to Reduce Verbatim Reproduction of Pre-training Data
• AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computation
• Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations
• People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text
• Echoes in AI: Quantifying lack of plot diversity in LLM outputs
• Does ChatGPT Have a Poetic Style?
• How Does the Disclosure of AI Assistance Affect the Perceptions of Writing?
• Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits
• Detection and Measurement of Syntactic Templates in Generated Text
• SHIELD: Evaluation and Defense Strategies for Copyright Compliance in LLM Text Generation
• AI and the Everyday Writer
• Art or Artifice? Large Language Models and the False Promise of Creativity
• Self-Alignment with Instruction Backtranslation
• Experimental evidence on the productivity effects of generative artificial intelligence
• Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks
• The debate over understanding in AI’s large language models
• The Goodreads “Classics”: A Computational Study of Readers, Amazon, and Crowdsourced Amateur Criticism
• Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models
• Working with AI: Measuring the Applicability of Generative AI to Occupations
• OpenAI o1 System Card

Discussions

• I wish, wish, wish, wish you were correct. And I still hope you are! But most studies – even of AI "literature" – find readers can't tell diff. And AI just gonna get better. Sigh. Eg arxiv.org/abs/251 [bsky, 3 points, 2 comments]
• Research Article (preprint): Readers Prefer Outputs of #AI Trained on Copyrighted #Books Over Expert Human #Writers arxiv.org/abs/2510.13939 #copyright #LLMs #GenAI [bsky, 3 points, 0 comments]
• Link to our paper : www.arxiv.org/abs/2510.13939 [bsky, 2 points, 0 comments]
• ai-generated content: entrepreneurs are choosing machine outputs over traditional human writers for business success arxiv.org [bsky, 0 points, 0 comments]
• Today in Everything is Terrible and What's the Point of Doing Anything? arxiv.org/abs/2510.13939 [bsky, 0 points, 0 comments]
• Not sure if this is the study referenced, but this one comes to the same conclusions: arxiv.org/abs/2510.13939 [bsky, 0 points, 1 comments]

Related

• Can Good Writing Be Generative? Expert-Level AI Writing Emerges through Fine-Tuning on High-Quality Books
• Everyone prefers human writers, including AI
• The Reader is the Metric: How Textual Features and Reader Profiles Explain Conflicting Evaluations of AI Creative Writing
• The artificial intelligence disclosure penalty: Humans persistently devalue AI-generated creative writing.
• Generative AI floods and dilutes the market for books