Mercury: Ultra-Fast Language Models Based on Diffusion

2025/06/17 by Inception Labs, Samar Khanna, Labs, Inception +23 · 26 voices · 85 citations
#cs.CL #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2506.17298

Abstract

We present Mercury, a new generation of commercial-scale large language models (LLMs) based on diffusion. These models are parameterized via the Transformer architecture and trained to predict multiple tokens in parallel. In this report, we detail Mercury Coder, our first set of diffusion LLMs designed for coding applications. Currently, Mercury Coder comes in two sizes: Mini and Small. These models set a new state-of-the-art on the speed-quality frontier. Based on independent evaluations conducted by Artificial Analysis, Mercury Coder Mini and Mercury Coder Small achieve state-of-the-art throughputs of 1109 tokens/sec and 737 tokens/sec, respectively, on NVIDIA H100 GPUs and outperform speed-optimized frontier models by up to 10x on average while maintaining comparable quality. We discuss additional results on a variety of code benchmarks spanning multiple languages and use-cases as well as real-world validation by developers on Copilot Arena, where the model currently ranks second on quality and is the fastest model overall. We also release a public API at https://platform.inceptionlabs.ai/ and free playground at https://chat.inceptionlabs.ai

Citations

• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• RWKV: Reinventing RNNs for the Transformer Era
• Training language models to follow instructions with human feedback
• Program Synthesis with Large Language Models
• Language Models are Few-Shot Learners
• Attention Is All You Need

Cited by

• Gumbel Distillation for Parallel Text Generation
• Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models
• LaCache: Exact Caching and Precision-Adaptive Inference for Diffusion Large Language Models
• FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models
• Induction in Both Directions: A Mechanistic Analysis of In-Context Learning in Masked Diffusion Language Models
• Trace-Based On-Policy Distillation for Masked Diffusion Language Models
• Team DACTYL at PAN 2026: Bayesian Data Mixing and Empirical X-risk Minimization for AI-text Detection
• A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models
• A Survey on Diffusion Language Models
• Large Language Diffusion Models
• dUltra: Ultra-Fast Diffusion Language Models via Reinforcement Learning
• Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbone
• Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion
• Rethinking the Generation Order of Block Diffusion Language Models
• Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs
• Fast and Accurate Causal Parallel Decoding using Jacobi Forcing
• Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
• Decoding Large Language Diffusion Models with Foreseeing Movement
• Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules
• GeoDiT: A Diffusion-based Vision-Language Model for Geospatial Understanding
• Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models
• From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models
• CDLM: Consistency Diffusion Language Models For Faster Sampling
• Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Models
• PartDiffuser: Part-wise 3D Mesh Generation via Discrete Diffusion
• A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models
• DiffuGR: Generative Document Retrieval with Diffusion Language Models
• Optimal Inference Schedules for Masked Diffusion Models
• Diffusion Language Models are Super Data Learners
• Watermarking Discrete Diffusion Language Models
• Diffuse Thinking: Exploring Diffusion Language Models as Efficient Thought Proposers for Reasoning
• From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models
• Aligning Diffusion Language Models via Unpaired Preference Optimization
• Attention Sinks in Diffusion Language Models
• From Masks to Worlds: A Hitchhiker's Guide to World Models
• How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practices
• Soft-Masked Diffusion Language Models
• Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Model
• Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Models
• Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief States
• On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks
• SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models
• Diffusion-Inspired Masked Fine-Tuning for Knowledge Injection in Autoregressive LLMs
• Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimization
• Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
• Accelerating Diffusion LLM Inference via Local Determinism Propagation
• DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning
• Fine-Tuning Masked Diffusion for Provable Self-Correction
• ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMs
• Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models
• Rainbow Padding: Mitigating Early Termination in Instruction-Tuned Diffusion LLMs
• Syntax-Guided Diffusion Language Models with User-Integrated Personalization
• Toward Safer Diffusion Language Models: Discovery and Mitigation of Priming Vulnerability
• Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models
• dParallel: Learnable Parallel Decoding for dLLMs
• RFG: Test-Time Scaling for Diffusion Large Language Model Reasoning with Reward-Free Guidance
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models
• Watermarking Diffusion Language Models
• TR2-D2: Tree Search Guided Trajectory-Aware Fine-Tuning for Discrete Diffusion
• Ultra-Fast Language Generation via Discrete Diffusion Divergence Instruct
• Alternatives To Next Token Prediction In Text Generation -- A Survey
• DiffuTester: Accelerating Unit Test Generation for Diffusion LLMs via Mining Structural Pattern
• Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Step
• Training Optimal Large Diffusion Language Models
• A2D: Any-Order, Any-Step Safety Alignment for Diffusion Language Models
• Tree Reward-Aligned Search for TReASURe in Masked Diffusion Language Models
• CoDA: Coding LM via Diffusion Adaptation
• Enabling Approximate Joint Sampling in Diffusion LMs
• d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation
• FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models
• Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokens
• Where and When to Commit: Candidate-Aware Decoding for Diffusion Language Models
• Beyond Autoregression: An Empirical Study of Diffusion Large Language Models for Code Generation
• Inpainting-Guided Policy Optimization for Diffusion Large Language Models
• Set Block Decoding is a Language Model Inference Accelerator
• Diffusion Generative Models Meet Compressed Sensing, with Applications to Imaging and Finance
• Dream-Coder 7B: An Open Diffusion Language Model for Code
• Reward-Weighted Sampling: Enhancing Non-Autoregressive Characteristics in Masked Diffusion LLMs
• Any-Order Flexible Length Masked Diffusion
• Diffusion Language Models Know the Answer Before Decoding
• Dream 7B: Diffusion Large Language Models
• PC-Sampler: Position-Aware Calibration of Decoding Bias in Masked Diffusion Models
• MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Models
• Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
• Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference
• Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Models

Discussions

• Mercury: Ultra-Fast Language Models Based on Diffusion [hn, 10 points, 2 comments]
• Mercury: Ultra-Fast Language Models Based on Diffusion [lobsters, 7 points, 0 comments]
• A new toy! arxiv.org/pdf/2506.17298 [bsky, 1 points, 0 comments]
• 最近AI初创公司Inception Labs推出的Mercury太牛了！基于扩散模型，专为聊天应用设计，速度超绝，比GPT-4.1 Nano等快7倍以上 Stefano Ermon参与开发，性能和效率领先。实时语音翻译等表现出色，虽生成质量有待提升，但已能在相关平台体验，感兴趣的伙伴可以看一下论文 https://arxiv.org/pdf/2506.17298 [bsky, 1 points, 0 comments]
• ⚡ Hackernews Top story: Mercury: Ultra-Fast Language Models Based on Diffusion [bsky, 1 points, 0 comments]
• next token prediction işini paralelize etmeyi başarmışlar. [bsky, 1 points, 0 comments]
• "Mercury: Ultra-Fast Language Models Based on Diffusion" New AI language models like Mercury are super fast, but they create problems in testing speeds for code. Getting our tests to keep up is really [bsky, 0 points, 0 comments]
• Mercury: Ultra-Fast Language Models Based on Diffusion [bsky, 0 points, 0 comments]
• Mercury: Ultra-Fast Language Models Based on Diffusion #HackerNews https://arxiv.org/abs/2506.17298 [bsky, 0 points, 0 comments]
• Mercury: Ultra-Fast Language Models Based on Diffusion https://arxiv.org/abs/2506.17298 https://news.ycombinator.com/item?id=44489690 [bsky, 0 points, 0 comments]
• Mercury: Ultra-fast language models based on diffusion https://arxiv.org/abs/2506.17298 arxiv.org [bsky, 0 points, 0 comments]
• Mercury: Ultra-Fast Language Models Based on Diffusion https://arxiv.org/abs/2506.17298 [bsky, 0 points, 0 comments]
• Mercury: Ultra-fast language models based on diffusion View Article | Join the HN Conversation Summary of HN discussion 🧵👇 #hacker-news [bsky, 0 points, 1 comments]
• 📰 Mercury: Ultra-Fast Language Models Based on Diffusion 🔗 https://arxiv.org/abs/2506.17298 💬 Discuss on HN [bsky, 0 points, 0 comments]
• 🌐Mercury: 拡散に基づく超高速言語モデル https://arxiv.org/abs/2506.17298 via #HackerNews [bsky, 0 points, 0 comments]
• Mercury: Ultra-Fast Language Models Based on Diffusion https://lobste.rs/s/fcnm2x #ai [bsky, 0 points, 0 comments]
• Mercury: Ultra-fast language models based on diffusion https://arxiv.org/abs/2506.17298 [bsky, 0 points, 0 comments]
• Mercury: Ultra-fast language models based on diffusion https://arxiv.org/abs/2506.17298 (https://news.ycombinator.com/item?id=44489690) [bsky, 0 points, 0 comments]
• Mercury: Ultra-Fast Language Models Based on Diffusion https://arxiv.org/abs/2506.17298 (http://news.ycombinator.com/item?id=44489690) [bsky, 0 points, 0 comments]
• Mercury: Ultra-Fast Language Models Based on Diffusion https://arxiv.org/abs/2506.17298 (http://news.ycombinator.com/item?id=44489690) [bsky, 0 points, 0 comments]
• Mercury: Ultra-fast language models based on diffusion https://arxiv.org/abs/2506.17298 (http://news.ycombinator.com/item?id=44489690) [bsky, 0 points, 0 comments]
• Mercury: Ultra-Fast Language Models Based on Diffusion https://arxiv.org/abs/2506.17298 (https://news.ycombinator.com/item?id=44489690) [bsky, 0 points, 0 comments]
• https://arxiv.org/abs/2506.17298 Diffusionモデルに基づく超高速言語モデルに関する研究。 Mercuryという新しいアーキテクチャを提案。 既存モデルよりも高速な推論が可能。 [bsky, 0 points, 0 comments]
• Mercury: Ultra-Fast Language Models Based on Diffusion view on hacker news [bsky, 0 points, 0 comments]
• Mercury: Ultra-Fast Language Models Based on Diffusion https://arxiv.org/abs/2506.17298 (https://news.ycombinator.com/item?id=44489690) [bsky, 0 points, 0 comments]
• https://bsky.app/profile/buzzing.cc.web.brid.gy/post/3ltfbn637eex2 [bsky, 0 points, 0 comments]

Related

• Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference
• Dream 7B: Diffusion Large Language Models
• dInfer: An Efficient Inference Framework for Diffusion Language Models
• LLaDA2.0: Scaling Up Diffusion Language Models to 100B
• Large Language Diffusion Models