Large Language Diffusion Models

2025/02/14 by Shen Nie, Fengqi Zhu, Nie, Shen +17 · 19 voices · 247 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.2502.09992

openalex publication_date 2025/02/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The capabilities of large language models (LLMs) are widely regarded as relying on autoregressive models (ARMs). We challenge this notion by introducing LLaDA, a diffusion model trained from scratch under the pre-training and supervised fine-tuning (SFT) paradigm. LLaDA employs a forward data masking process and a reverse generation process, parameterized by a Transformer to predict masked tokens. It provides a principled generative approach for probabilistic inference by optimizing a likelihood lower bound. Across extensive benchmarks on general tasks, math, code, and so on, LLaDA demonstrates strong scalability and performs comparably to our self-constructed ARM baselines. Remarkably, LLaDA 8B is competitive with strong LLMs like LLaMA3 8B in in-context learning and, after SFT, exhibits impressive instruction-following abilities in case studies such as multi-turn dialogue. Moreover, LLaDA addresses the reversal curse, surpassing GPT-4o in a reversal poem completion task. Our findings show the promise of diffusion models for language modeling at scale and challenge the common assumption that core LLM capabilities discussed above inherently depend on ARMs. Project page and codes: https://ml-gsai.github.io/LLaDA-demo/.

Citations

• Mercury: Ultra-Fast Language Models Based on Diffusion
• Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
• Magicoder: Empowering Code Generation with OSS-Instruct
• Mistral 7B
• The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
• Language Modeling Is Compression
• Bayesian Flow Networks
• Generative Agents: Interactive Simulacra of Human Behavior
• Training Compute-Optimal Large Language Models
• Training language models to follow instructions with human feedback
• Program Synthesis with Large Language Models
• Diffusion Models Beat GANs on Image Synthesis
• Language Models are Few-Shot Learners
• Attention Is All You Need

Cited by

• Context-weighted Discrete Flow Matching
• LaViDa: A Large Diffusion Language Model for Multimodal Understanding
• Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models
• Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation
• Dimension-Free Convergence of Discrete Diffusion Models: Adjoint Equations Induce the Right Space
• Multi-Mask Diffusion Language Models for Few-Step Generation
• Tensor-Train Joint Modeling for Few-Step Discrete Diffusion
• One-step lowest-variance selection in a Gaussian random-field model motivated by masked diffusion: Total correlation and a square root collision threshold
• ST-Veto: Spatio-Temporal Token Veto for Diffusion MLLMs via Taylor Prediction and Visual Grounding
• UnMaskFork: Test-Time Scaling for Masked Diffusion via Deterministic Action Branching
• Breaking the Factorization Barrier in Diffusion Language Models
• A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models
• Mask-Aware Policy Gradients for Diffusion Language Models
• UNIFUSION: Adapting Autoregressive Language Models into Discrete Diffusion under a Unified Reverse-Rate Objective
• DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding
• JUMP: Single-Pass Membership Inference on Fine-Tuned Diffusion Language Models
• Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs
• Token Time Continuous Diffusion for Language Modeling
• Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL
• Speculative Speculative Decoding
• Mind captioning: Evolving descriptive text of mental content from human brain activity
• LLaDA-MoE: A Sparse MoE Diffusion Language Model
• A Survey on Diffusion Language Models
• Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential
• Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
• Steering Generative Models with Experimental Data for Protein Fitness Optimization
• Neurosymbolic Diffusion Models
• Idiosyncrasies in Large Language Models
• dUltra: Ultra-Fast Diffusion Language Models via Reinforcement Learning
• Plug-and-Play Fidelity Optimization for Diffusion Transformer Acceleration via Cumulative Error Minimization
• WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference
• Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbone
• Self-Evaluation Unlocks Any-Step Text-to-Image Generation
• Co-GRPO: Co-Optimized Group Relative Policy Optimization for Masked Diffusion Model
• DiRL: An Efficient Post-Training Framework for Diffusion Language Models
• Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion
• PreDiff-LM: Pretrained Discrete Masked Diffusion Language Modeling with Hybrid Attention
• TRE: Training-Free Hallucination Detection for Diffusion Language Models
• Beyond Block Boundaries: Multi-Block Editing for Diffusion Large Language Models
• CaRE Compute-aware Remasking Evaluation Protocol for Masked Diffusion Language Models
• Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty
• ReaSeq: Unleashing World Knowledge via Reasoning for Sequential Modeling
• Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs
• PairFlow: Closed-Form Source-Target Coupling for Few-Step Generation in Discrete Flow Models
• dMLLM-TTS: Self-Verified and Efficient Test-Time Scaling for Diffusion Multi-Modal Large Language Models
• DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Models
• DEER: Draft with Diffusion, Verify with Autoregressive Models
• HD-Prot: A Protein Language Model for Joint Sequence-Structure Modeling with Continuous Structure Tokens
• Fast and Accurate Causal Parallel Decoding using Jacobi Forcing
• Dual-objective Language Models: Training Efficiency Without Overfitting
• ViewMask-1-to-3: Multi-View Consistent Image Generation via Multimodal Discrete Diffusion Models
• Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
• ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding
• LINA: Learning INterventions Adaptively for Physical Alignment and Generalization in Diffusion Models
• MineTheGap: Automatic Mining of Biases in Text-to-Image Models
• Scaling Behavior of Discrete Diffusion Language Models
• d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models
• Learning Unmasking Policies for Diffusion Language Models
• Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs
• From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs
• WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Driving
• WAM-Flow: Parallel Coarse-to-Fine Motion Planning via Discrete Flow Matching for Autonomous Driving
• Fast SceneScript: Accurate and Efficient Structured Language Model via Multi-Token Prediction
• Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective
• dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoning
• Self-Paced and Self-Corrective Masked Prediction for Movie Trailer Generation
• Decoding Large Language Diffusion Models with Foreseeing Movement
• PretrainZero: Reinforcement Active Pretraining
• Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules
• Diffusion-Augmented Markov Decision Processes for Maximum Entropy Reinforcement Learning
• TUNA: Taming Unified Visual Representations for Native Unified Multimodal Models
• Syndrome-Flow Consistency Model Achieves One-step Denoising Error Correction Codes
• Accelerating Inference of Masked Image Generators via Reinforcement Learning
• MM-ACT: Learn from Multimodal Parallel Generation to Act
• EDIT: Early Diffusion Inference Termination for dLLMs Based on Dynamics of Training Gradients
• Markovian Scale Prediction: A New Era of Visual Autoregressive Generation
• Closed-Loop Transformers: Autoregressive Modeling as Iterative Latent Equilibrium
• E₀: Enhancing Generalization and Fine-Grained Control in VLA Models via Continuized Discrete Diffusion
• Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models
• Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models
• From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models
• CDLM: Consistency Diffusion Language Models For Faster Sampling
• Fluid Intelligence: A Forward Look on AI Foundation Models in Computational Fluid Dynamics
• Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Models
• FlowSteer: Guiding Few-Step Image Synthesis with Authentic Trajectories
• PartDiffuser: Part-wise 3D Mesh Generation via Discrete Diffusion
• DELTA: Language Diffusion-based EEG-to-Text Architecture
• Demystifying Diffusion Objectives: Reweighted Losses are Better Variational Bounds
• Bringing Stability to Diffusion: Decomposing and Reducing Variance of Training Masked Diffusion Models
• WavefrontDiffusion: Dynamic Decoding Schedule for Improved Reasoning
• Reasoning in Diffusion Large Language Models is Concentrated in Dynamic Confusion Zones
• A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models
• AsyncVLA: Asynchronous Flow Matching for Vision-Language-Action Models
• MMaDA-Parallel: Multimodal Large Diffusion Language Models for Thinking-Aware Editing and Generation
• KLASS: KL-Guided Fast Inference in Masked Diffusion Models
• Optimal Inference Schedules for Masked Diffusion Models
• Forget BIT, It is All about TOKEN: Towards Semantic Information Theory for LLMs
• Diffusion Language Models are Super Data Learners
• Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models
• Effective Test-Time Scaling of Discrete Diffusion through Iterative Refinement
• Watermarking Discrete Diffusion Language Models
• The Era of Agentic Organization: Learning to Organize with Language Models
• Don't Let It Fade: Preserving Edits in Diffusion Language Models via Token Timestep Allocation
• PairUni: Pairwise Training for Unified Multimodal Language Models
• From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models
• Speech-XL: Towards Long-Form Speech Understanding in Large Speech Language Models
• Information-Theoretic Discrete Diffusion
• Variational Masked Diffusion Models
• Simple Denoising Diffusion Language Models
• Encoder-Decoder Diffusion Language Models for Efficient Training and Inference
• Aligning Diffusion Language Models via Unpaired Preference Optimization
• Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing
• Efficient semantic uncertainty quantification in language models via diversity-steered sampling
• Noise is All You Need: Solving Linear Inverse Problems by Noise Combination Sampling with Diffusion Models
• From Masks to Worlds: A Hitchhiker's Guide to World Models
• A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generation
• Loopholing Discrete Diffusion: Deterministic Bypass of the Sampling Wall
• From Denoising to Refining: A Corrective Framework for Vision-Language Diffusion Model
• No Compute Left Behind: Rethinking Reasoning and Sampling with Masked Diffusion Models
• How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practices
• Latent-Augmented Discrete Diffusion Models
• Planned Diffusion
• Soft-Masked Diffusion Language Models
• Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Model
• Planner and Executor: Collaboration between Discrete Diffusion And Autoregressive Models in Reasoning
• Attention Is All You Need for KV Cache in Diffusion LLMs
• On the Reasoning Abilities of Masked Diffusion Language Models
• NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching
• DiSTAR: Diffusion over a Scalable Token Autoregressive Representation for Speech Generation
• Test-Time Anchoring for Discrete Diffusion Posterior Sampling
• Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief States
• UltraLLaDA: Scaling the Context Length to 128K for Diffusion Large Language Models
• On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks
• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs
• Beyond Surface Reasoning: Unveiling the True Long Chain-of-Thought Capacity of Diffusion Large Language Models
• SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models
• Diffusion-Inspired Masked Fine-Tuning for Knowledge Injection in Autoregressive LLMs
• dInfer: An Efficient Inference Framework for Diffusion Language Models
• Guided Star-Shaped Masked Diffusion
• Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimization
• Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
• Accelerating Diffusion LLM Inference via Local Determinism Propagation
• DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning
• OBS-Diff: Accurate Pruning For Diffusion Models in One-Shot
• AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs
• Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understanding
• SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation
• Discrete Diffusion Models with MLLMs for Unified Medical Multimodal Generation
• Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policies
• Fine-Tuning Masked Diffusion for Provable Self-Correction
• Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models
• Test-Time Scaling in Diffusion LLMs via Hidden Semi-Autoregressive Experts
• ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMs
• AtomWorld: A Benchmark for Evaluating Spatial Reasoning in Large Language Models on Crystalline Materials
• Asynchronous Denoising Diffusion Models for Aligning Text-to-Image Generation
• LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning
• Demystifying MaskGIT Sampler and Beyond: Adaptive Order Selection in Masked Diffusion
• Self Speculative Decoding for Diffusion Large Language Models
• Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models
• Rainbow Padding: Mitigating Early Termination in Instruction-Tuned Diffusion LLMs
• OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flows
• DMark: Order-Agnostic Watermarking for Diffusion Large Language Models
• Consolidating Reinforcement Learning for Multimodal Discrete Diffusion Models
• Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner
• In-Situ Tweedie Discrete Diffusion Models
• Syntax-Guided Diffusion Language Models with User-Integrated Personalization
• Toward Safer Diffusion Language Models: Discovery and Mitigation of Priming Vulnerability
• Training Large Language Models To Reason In Parallel With Global Forking Tokens
• Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling
• Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models
• dParallel: Learnable Parallel Decoding for dLLMs
• AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Size
• Data-to-Energy Stochastic Dynamics
• Fast-dLLM v2: Efficient Block-Diffusion LLM
• dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thought
• TraceDet: Hallucination Detection from the Decoding Trace of Diffusion Large Language Models
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models
• Watermarking Diffusion Language Models
• TR2-D2: Tree Search Guided Trajectory-Aware Fine-Tuning for Discrete Diffusion
• Ultra-Fast Language Generation via Discrete Diffusion Divergence Instruct
• Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding
• Sequential Diffusion Language Models
• Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Step
• DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding
• Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models
• Training Optimal Large Diffusion Language Models
• RIV: Recursive Introspection Mask Diffusion Vision Language Model
• Planner Aware Path Learning in Diffusion Language Models Training
• A2D: Any-Order, Any-Step Safety Alignment for Diffusion Language Models
• Tree Reward-Aligned Search for TReASURe in Masked Diffusion Language Models
• CoDA: Coding LM via Diffusion Adaptation
• Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models
• d²Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
• Reinforcement Learning with Discrete Diffusion Policies for Combinatorial Action Spaces
• Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddings
• CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning
• Scale-Wise VAR is Secretly Discrete Diffusion
• Decoding quantum low density parity check codes with diffusion
• Generation Properties of Stochastic Interpolation under Finite Training Set
• Discrete Guidance Matching: Exact Guidance for Discrete Flow Matching
• Error Analysis of Discrete Flow with Generator Matching
• On the Complexity Theory of Masked Discrete Diffusion: From poly(1/ε) to Nearly ε-Free
• Enabling Approximate Joint Sampling in Diffusion LMs
• Deterministic Discrete Denoising
• d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation
• FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models
• Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving
• From Text to Talk: Audio-Language Model Needs Non-Autoregressive Joint Training
• Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokens
• PIRF: Physics-Informed Reward Fine-Tuning for Diffusion Models
• Where and When to Commit: Candidate-Aware Decoding for Diffusion Language Models
• ELF: Embedded Language Flows
• Dream-Cubed: Controllable Generative Modeling in Minecraft by Training on Billions of Cubes
• Visual Instruction Pretraining for Domain-Specific Foundation Models
• Audio-Conditioned Diffusion LLMs for ASR and Deliberation Processing
• Diffusion-Based Cross-Modal Feature Extraction for Multi-Label Classification
• Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning
• Masked Diffusion Models as Energy Minimization
• DSFT: Inspiring Diffusion Large Language Models to Comprehend Mathematical and Logical Patterns
• LLaDA-VLA: Vision Language Diffusion Action Models
• COGITAO: A Visual Reasoning Framework To Study Compositionality & Generalization
• Set Block Decoding is a Language Model Inference Accelerator
• Diffusion Generative Models Meet Compressed Sensing, with Applications to Imaging and Finance
• Dream-Coder 7B: An Open Diffusion Language Model for Code
• Any-Order Flexible Length Masked Diffusion
• Tabular Diffusion Counterfactual Explanations
• Universal Properties of Activation Sparsity in Modern Large Language Models
• Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies
• Diffusion Language Models Know the Answer Before Decoding
• Blockwise SFT for Diffusion Language Models: Reconciling Bidirectional Attention and Autoregressive Decoding
• Rethinking Human-Object Interaction Evaluation for both Vision-Language Models and HOI-Specific Methods
• Dream 7B: Diffusion Large Language Models
• Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs
• DPad: Efficient Diffusion Language Models with Suffix Dropout
• Reinforced Context Order Recovery for Adaptive Reasoning and Planning
• ViLaD: A Large Vision Language Diffusion Framework for End-to-End Autonomous Driving
• MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Models
• Navigating the Exploration-Exploitation Tradeoff in Inference-Time Scaling of Diffusion Models
• Improving Text Style Transfer using Masked Diffusion Language Models with Inference-time Scaling
• MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Control
• Constrained Decoding of Diffusion LLMs with Context-Free Grammars
• Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models
• Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
• One Model for All: Unified Try-On and Try-Off in Any Pose via LLM-Inspired Bidirectional Tweedie Diffusion
• Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference
• LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understanding
• Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Models

Discussions

• what the fuck shouting "ENHANCE" repeatedly at a next word predictor gets you much more rounded synthetic intellect scores arxiv.org/pdf/2502.09992 [bsky, 25 points, 7 comments]
• LLaDA - Large Language Diffusion Models arxiv.org/abs/2502.09992 Significant progress towards language diffusion models. Reportedly on par with LLaMA3 on many benchmarks. [bsky, 9 points, 2 comments]
• Wow Meta trained a version of EvoDiff for natural language! arxiv.org/abs/2502.09992 [bsky, 7 points, 0 comments]
• Large Language Diffusion Models [hn, 7 points, 3 comments]
• arxiv.org/abs/2502.09992 [bsky, 4 points, 0 comments]
• Check out the paper here, it’s an interesting way of thinking: arxiv.org/pdf/2502.09992 [bsky, 4 points, 0 comments]
• Large Language Diffusion Models [lobsters, 3 points, 0 comments]
• Interesting tid bit, uniformly sampling t then doing coinflips at prob t per step ala LLaDA arxiv.org/abs/2502.09992 is effectively balancing over all possible masks. See bottom of this post from Tim [bsky, 3 points, 1 comments]
• Large Language Diffusion Models [hn, 2 points, 0 comments]
• Diffusion LLM Has Arrived [hn, 2 points, 0 comments]
• Large Language Diffusion Models [hn, 2 points, 0 comments]
• A group of Chinese researchers have developed this model called 'LLaDA' or Large Language Diffusion model. Their 8b parameter model is on par with much larger LLM's. paper: arxiv.org/pdf/2502.09992 [bsky, 1 points, 1 comments]
• Yes ... and this visualization of the diffusion model's "thinking" is perhaps much more honest than the verbalization or inner dialog we see with exposed 01, 03 etc reasoning traces. arxiv.org/abs/25 [bsky, 1 points, 0 comments]
• Learn more here ➡️ arxiv.org/abs/2502.09992 [bsky, 0 points, 0 comments]
• If it’s Thursday, I’m at ADA (thanks Gordon!) for a solid talk. arxiv.org/abs/2502.09992 Thanks to @suhaspai.bsky.social for organizing! [bsky, 0 points, 0 comments]
• 2 links about LLdMs paper: arxiv.org/pdf/2502.09992 startup: www.inceptionlabs.ai (they are independent) [bsky, 0 points, 0 comments]
• And here is the original paper, published a few weeks ago. It comes with an open-weight 8B model, ready to test on HugginFace. buff.ly/0D00Al1 [bsky, 0 points, 0 comments]
• arxiv.org/abs/2502.09992 [bsky, 0 points, 0 comments]
• Large Language Diffusion Models https://lobste.rs/s/ibx3dt #ai [bsky, 0 points, 0 comments]

Related

• Scaling up Masked Diffusion Models on Text
• Dream 7B: Diffusion Large Language Models
• Beyond Autoregression: Fast LLMs via Self-Distillation Through Time
• LLaDA2.0: Scaling Up Diffusion Language Models to 100B
• Diffusion-Inspired Masked Fine-Tuning for Knowledge Injection in Autoregressive LLMs