Attributing authorship via the perplexity of authorial language models

2025/07/03 by Weihang Huang, Akira Murakami, Jack Grieve · 1 voice · 2 citations
Computer Science · #Authorship Attribution and Profiling #Topic Modeling #Hate Speech and Cyberbullying Detection

paper · doi:10.1371/journal.pone.0327081

openalex publication_date 2025/07/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/06/11

Abstract

Authorship attribution is the task of identifying the most likely author of a questioned document from a set of candidate authors, where each candidate is represented by a writing sample. A wide range of quantitative methods for inferring authorship have been developed in stylometry, but the rise of Large Language Models (LLMs) offers new opportunities in this field. In this paper, we introduce a technique for authorship attribution based on fine-tuned LLMs. Our approach involves first further pretraining LLMs for each candidate author based on their known writings and then assigning the questioned document to the author whose Authorial Language Model (ALM) finds the questioned document most predictable, measured as the perplexity of the questioned document. We find that our approach meets or exceeds the current state-of-the-art on several standard benchmarking datasets. In addition, we show how our approach can be used to measure the predictability of each word in a questioned document for a given candidate ALM, allowing the linguistic patterns that drive our attributions to be inspected directly. Finally, we analyze what types of words generally drive successful attributions, finding that content words classes are characterized by a higher density of authorship information than function word classes, challenging a long-standing assumption of stylometry.

Citations

• The Sociolinguistic Foundations of Language Modeling
• LLMDet: A Third Party Large Language Models Generated Text Detection Tool
• CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters
• Don't Stop Pretraining: Adapt Language Models to Domains and Tasks

Cited by

• A Stylometric Application of Large Language Models
• LLM one-shot style transfer for Authorship Attribution and Verification

Discussions

• Our new PLOS ONE paper, led by a PhD student co-supervised by @jwgrieve.bsky.social and me, is out! We fine-tuned LLMs per author to create Authorial Language Models, attributing texts via perplexity. [bsky, 4 points, 0 comments]

Related

• ALMs: Authorial Language Models for Authorship Attribution
• Can Large Language Models Identify Authorship?
• A Bayesian Approach to Harnessing the Power of LLMs in Authorship Attribution
• Neural Authorship Attribution: Stylometric Analysis on Large Language Models
• LLM one-shot style transfer for Authorship Attribution and Verification