Watermarking Diffusion Language Models

2025/09/29 by Thibaud Gloaguen, Robin Staab, Gloaguen, Thibaud +5 · 2 citations
Computer Science · #Advanced Steganography and Watermarking Techniques #Artificial Intelligence (cs.AI) #Cellular Automata and Applications #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Internet Traffic Analysis and Secure E-voting #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2509.24368

openalex publication_date 2025/09/29 · openalex created_date 2025/10/19 · openalex updated_date 2026/07/28

Abstract

We introduce the first watermark tailored for diffusion language models (DLMs), an emergent LLM paradigm able to generate tokens in arbitrary order, in contrast to standard autoregressive language models (ARLMs) which generate tokens sequentially. While there has been much work in ARLM watermarking, a key challenge when attempting to apply these schemes directly to the DLM setting is that they rely on previously generated tokens, which are not always available with DLM generation. In this work we address this challenge by: (i) applying the watermark in expectation over the context even when some context tokens are yet to be determined, and (ii) promoting tokens which increase the watermark strength when used as context for other tokens. This is accomplished while keeping the watermark detector unchanged. Our experimental evaluation demonstrates that the DLM watermark leads to a >99% true positive rate with minimal quality impact and achieves similar robustness to existing ARLM watermarks, enabling for the first time reliable DLM watermarking.

Citations

• Can You Detect the Difference?
• Watermarking Autoregressive Image Generation
• Mercury: Ultra-Fast Language Models Based on Diffusion
• The Diffusion Duality
• LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning
• MMaDA: Multimodal Large Diffusion Language Models
• Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
• Mark Your LLM: Detecting the Misuse of Open-Source Large Language Models via Watermarking
• Large Language Diffusion Models
• Towards Watermarking of Open-Source LLMs
• Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions
• Large Language Models to Diffusion Finetuning
• GaussMark: A Practical Approach for Structural Watermarking of Language Models
• Provably Robust Watermarks for Open-Source Language Models
• A Watermark for Order-Agnostic Language Models
• Can Watermarked LLMs be Identified by Users via Crafted Prompts?
• Scalable watermarking for identifying large language model outputs
• Zero-Shot Detection of LLM-Generated Text using Token Cohesiveness
• Simple and Effective Masked Diffusion Language Models
• Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data
• Simplified and Generalized Masked Diffusion for Discrete Data
• Large Language Model Watermark Stealing With Mixed Integer Programming
• Black-Box Detection of Language Model Watermarks
• MarkLLM: An Open-Source Toolkit for LLM Watermarking
• Better & Faster Large Language Models via Multi-token Prediction
• Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models
• Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models
• Watermark Stealing in Large Language Models
• GPQA: A Graduate-Level Google-Proof Q&A Benchmark
• WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models
• The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
• Robust Distortion-free Watermarks for Language Models
• Three Bricks to Consolidate Watermarks for Large Language Models
• Provable Robust Watermarking for AI-Generated Text
• On the Reliability of Watermarks for Large Language Models
• Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust
• AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
• Reflected Diffusion Models
• The Stable Signature: Rooting Watermarks in Latent Diffusion Models
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• Cognitive Constraint Simulation and the Geometry of Human Authorship: A First-Principles Theory of AI Text Detection
• A Watermark for Large Language Models
• Training Verifiers to Solve Math Word Problems
• FinQA: A Dataset of Numerical Reasoning over Financial Data
• Program Synthesis with Large Language Models
• Structured Denoising Diffusion Models in Discrete State-Spaces
• Diffusion Models Beat GANs on Image Synthesis
• Improved Denoising Diffusion Probabilistic Models
• Denoising Diffusion Implicit Models
• Measuring Massive Multitask Language Understanding
• Denoising Diffusion Probabilistic Models
• Language Models are Few-Shot Learners
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• ELI5: Long Form Question Answering
• HellaSwag: Can a Machine Really Finish Your Sentence?
• Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
• Semi-supervised Sequence Learning
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Cited by

• Watermarking Discrete Diffusion Language Models
• DITTO: A Spoofing Attack Framework on Watermarked LLMs via Knowledge Distillation

Related

• DMark: Order-Agnostic Watermarking for Diffusion Large Language Models
• Watermarking Discrete Diffusion Language Models
• Watermarking Low-entropy Generation for Large Language Models: An Unbiased and Low-risk Method
• Watermarking Diffusion Model
• A Recipe for Watermarking Diffusion Models