Edit Flows: Flow Matching with Edit Operations

2025/06/10 by Marton Havasi, Havasi, Marton, Brian Karrer +5 · 1 voice · 21 citations
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Handwritten Text Recognition Techniques #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2506.09018

openalex publication_date 2025/06/10 · arxiv published 2025/06/10 · openalex created_date 2025/10/10 · arxiv updated 2025/11/12 · openalex updated_date 2026/07/28

Abstract

Autoregressive generative models naturally generate variable-length sequences, while non-autoregressive models struggle, often imposing rigid, token-wise structures. We propose Edit Flows, a non-autoregressive model that overcomes these limitations by defining a discrete flow over sequences through edit operations\unicodex2013insertions, deletions, and substitutions. By modeling these operations within a Continuous-time Markov Chain over the sequence space, Edit Flows enable flexible, position-relative generation that aligns more closely with the structure of sequence data. Our training method leverages an expanded state space with auxiliary variables, making the learning process efficient and tractable. Empirical results show that Edit Flows outperforms both autoregressive and mask models on image captioning and significantly outperforms the mask construction in text and code generation.

Citations

• Large Language Diffusion Models
• Flow Matching Guide and Code
• Flex Attention: A Programming Model for Generating Optimized Attention Kernels
• Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspective
• Generator Matching: Generative modeling with arbitrary Markov processes
• Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling
• The Llama 3 Herd of Models
• Discrete Flow Matching
• Simple and Effective Masked Diffusion Language Models
• Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data
• Simplified and Generalized Masked Diffusion for Discrete Data
• Unlocking Guidance for Discrete State-Space Diffusion and Flow Models
• σ-GPTs: A New Approach to Autoregressive Models
• Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
• LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
• Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design
• Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution
• Improved Baselines with Visual Instruction Tuning
• Code Llama: Open Foundation Models for Code
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• On Kinetic Optimal Probability Paths for Generative Models
• Trans-Dimensional Generative Modeling via Jump Diffusion Models
• Scalable Diffusion Models with Transformers
• Generating Sequences by Learning to Self-Correct
• DiffusER: Discrete Diffusion via Edit-based Reconstruction
• Classifier-Free Diffusion Guidance
• A Continuous Time Framework for Discrete Denoising Models
• COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics
• MaskGIT: Masked Generative Image Transformer
• CM3: A Causal Masked Multimodal Model of the Internet
• ClipCap: CLIP Prefix for Image Captioning
• Autoregressive Diffusion Models
• Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
• Structured Denoising Diffusion Models in Discrete State-Spaces
• Relative Positional Encoding for Transformers with Linear Complexity
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• Learning Transferable Visual Models From Natural Language Supervision
• Denoising Diffusion Probabilistic Models
• Imputer: Sequence Modelling via Imputation and Dynamic Programming
• Unified Vision-Language Pre-Training for Image Captioning and VQA
• XLNet: Generalized Autoregressive Pretraining for Language Understanding
• Levenshtein Transformer
• Insertion Transformer: Flexible Sequence Generation via Insertion Operations
• Insertion-based Decoding with automatically Inferred Generation Order
• CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling
• Lecture Notes: Temporal Point Processes and the Conditional Intensity Function
• Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement
• Attention Is All You Need
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• Microsoft COCO: Common Objects in Context
• A general method for numerically simulating the stochastic time evolution of coupled chemical reactions
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Cited by

• Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbone
• XLM: A Python package for non-autoregressive language models
• Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models
• Time dependent loss reweighting for flow matching and diffusion models is theoretically justified
• Branching Flows: Discrete, Continuous, and Manifold Flow Matching with Splits and Deletions
• A Diffusion Model to Shrink Proteins While Maintaining Their Function
• Guided Star-Shaped Masked Diffusion
• Edit-Based Flow Matching for Temporal Point Processes
• Drax: Speech Recognition with Discrete Flow Matching
• OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flows
• Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling
• DiffuTester: Accelerating Unit Test Generation for Diffusion LLMs via Mining Structural Pattern
• Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models
• Planner Aware Path Learning in Diffusion Language Models Training
• Discrete Guidance Matching: Exact Guidance for Discrete Flow Matching
• Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokens
• Exploring Synthesizable Chemical Space with Iterative Pathway Refinements
• Any-Order Flexible Length Masked Diffusion
• Syntax Repair as Language Intersection
• Unveiling the Potential of Diffusion Large Language Model in Controllable Generation
• Partition Generative Modeling: Masked Modeling Without Masks
• Insertion Language Models: Sequence Generation with Arbitrary-Position Insertions
• DFM-VLA: Iterative Action Refinement for Robot Manipulation via Discrete Flow Matching

Discussions

• (We also show this for Edit Flows, which lies just outside of Generator Matching) arxiv.org/abs/2506.09018 [bsky, 2 points, 1 comments]

Related

• Learning to Model Editing Processes
• Training-Free Text-Guided Image Editing with Visual Autoregressive Model
• Edit-Based Flow Matching for Temporal Point Processes
• OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flows
• Anchor Token Matching: Implicit Structure Locking for Training-free AR Image Editing