Rectified Flow: A Marginal Preserving Approach to Optimal Transport

2022/09/29 by Qiang Liu, Liu, Qiang · 1 voice · 106 citations
Computer Science · Engineering · Mathematics · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Applied mathematics #Computer science #Concave function #Convex function #Coupling (piping) #Differentiable function #Engineering #FOS: Computer and information sciences #Flow (mathematics) #Function (biology) #Geometry #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Marginal cost #Mathematical analysis #Mathematical optimization #Mathematics #Monotonic function #Regular polygon #Sequence (biology) #Set (abstract data type) #Stochastic Gradient Optimization Techniques #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2209.14577

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2022/09/29 · arxiv published 2022/09/29 · arxiv updated 2022/09/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present a flow-based approach to the optimal transport (OT) problem between two continuous distributions π₀,π₁ on ℝ^d, of minimizing a transport cost 𝔼[c(X₁-X₀)] in the set of couplings (X₀,X₁) whose marginal distributions on X₀,X₁ equals π₀,π₁, respectively, where c is a cost function. Our method iteratively constructs a sequence of neural ordinary differentiable equations (ODE), each learned by solving a simple unconstrained regression problem, which monotonically reduce the transport cost while automatically preserving the marginal constraints. This yields a monotonic interior approach that traverses inside the set of valid couplings to decrease the transport cost, which distinguishes itself from most existing approaches that enforce the coupling constraints from the outside. The main idea of the method draws from rectified flow, a recent approach that simultaneously decreases the whole family of transport costs induced by convex functions c (and is hence multi-objective in nature), but is not tailored to minimize a specific transport cost. Our method is a single-object variant of rectified flow that guarantees to solve the OT problem for a fixed, user-specified convex cost function c.

Cited by

• Plug-and-Play Fidelity Optimization for Diffusion Transformer Acceleration via Cumulative Error Minimization
• Generative Modeling by Value-Driven Transport
• Autoregressive One-Step Generative Modeling for Dynamical System Forecasting
• Loom: Diffusion-Transformer for Interleaved Generation
• PhysBrain: Human Egocentric Data as a Bridge from Vision Language Models to Physical Intelligence
• FALCON: Few-step Accurate Likelihoods for Continuous Flows
• Correspondence-Oriented Imitation Learning: Flexible Visuomotor Control with 3D Conditioning
• GR-RL: Going Dexterous and Precise for Long-Horizon Robotic Manipulation
• SPARK: Sim-ready Part-level Articulated Reconstruction with VLM Knowledge
• FlowEO: Generative Unsupervised Domain Adaptation for Earth Observation
• High-dimensional Mean-Field Games by Particle-based Flow Matching
• Flow Matching for Tabular Data Synthesis
• Overcoming the Curvature Bottleneck in MeanFlow
• E₀: Enhancing Generalization and Fine-Grained Control in VLA Models via Continuized Discrete Diffusion
• TReFT: Taming Rectified Flow Models For One-Step Image Translation
• SPEAR-1: Scaling Beyond Robot Demonstrations via 3D Understanding
• TimeFlow: Towards Stochastic-Aware and Efficient Time Series Generation via Flow Matching Modeling
• Physics-Informed Design of Input Convex Neural Networks for Consistency Optimal Transport Flow Matching
• MusRec: Zero-Shot Text-to-Music Editing via Rectified Flow and Diffusion Transformers
• Evo-1: Lightweight Vision-Language-Action Model with Preserved Semantic Alignment
• iFlyBot-VLA Technical Report
• Curly Flow Matching for Learning Non-gradient Field Dynamics
• Autoregressive Boltzmann Generators
• PromptReverb: Multimodal Room Impulse Response Generation Through Latent Rectified Flow Matching
• HollowFlow: Efficient Sample Likelihood Evaluation using Hollow Message Passing
• GigaBrain-0: A World Model-Powered Vision-Language-Action Model
• Gradient Variance Reveals Failure Modes in Flow-Based Generative Models
• UniRL-Zero: Reinforcement Learning on Unified Models with Joint Language Model and Diffusion Model Experts
• One-step Diffusion Models with Bregman Density Ratio Matching
• LeapFactual: Reliable Visual Counterfactual Explanation Using Conditional Flow Matching
• Expertise need not monopolize: Action-Specialized Mixture of Experts for Vision-Language-Action Learning
• pi-Flow: Policy-Based Few-Step Generation via Imitation Distillation
• Adapting Noise to Data: Generative Flows from 1D Processes
• Hierarchical Koopman Diffusion: Fast Generation with Interpretable Diffusion Trajectory
• An Eulerian Perspective on Straight-Line Sampling
• Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modeling
• Fine-Tuning Flow Matching via Maximum Likelihood Estimation of Reconstructions
• Asymmetric Flow Models
• TC-LoRA: Temporally Modulated Conditional LoRA for Adaptive Diffusion Control
• Speculative Jacobi-Denoising Decoding for Accelerating Autoregressive Text-to-image Generation
• Goal-oriented Backdoor Attack against Vision-Language-Action Models via Physical Objects
• Deep Neural Networks Inspired by Differential Equations
• Rectified-CFG++ for Flow Based Models
• Carré du champ flow matching: better quality-generalisation tradeoff in generative models
• Contrastive Representation Regularization for Vision-Language-Action Models
• Riemannian Consistency Model
• Align Your Tangent: Training Better Consistency Models via Manifold-Aligned Tangents
• F-scheduler: illuminating the free-lunch design space for fast sampling of diffusion models
• FLOWER: A Flow-Matching Solver for Inverse Problems
• Flow Matching with Semidiscrete Couplings
• OAT-FM: Optimal Acceleration Transport for Improved Flow Matching
• FlashI2V: Fourier-Guided Latent Shifting Prevents Conditional Image Leakage in Image-to-Video Generation
• Diffusion Bridge or Flow Matching? A Unifying Framework and Comparative Analysis
• Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddings
• Universal Inverse Distillation for Matching Models with Real-Data Supervision (No GANs)
• Developing Vision-Language-Action Model from Egocentric Videos
• Score-based Idempotent Distillation of Diffusion Models
• Federated Flow Matching
• Flow Straight and Fast in Hilbert Space: Functional Rectified Flow
• LADB: Latent Aligned Diffusion Bridges for Semi-Supervised Domain Translation
• Provable Mixed-Noise Learning with Flow-Matching
• Unsupervised Real-World Super-Resolution via Rectified Flow Degradation Modelling
• villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Models
• H-RDT: Human Manipulation Enhanced Bimanual Robotic Manipulation
• GR-3 Technical Report
• From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models
• Hierarchical Rectified Flow Matching with Mini-Batch Couplings
• Protenix-Mini: Efficient Structure Predictor via Compact Architecture, Few-Step Diffusion and Switchable pLM
• Straighten Viscous Rectified Flow via Noise Optimization
• Flow Diverse and Efficient: Learning Momentum Flow Matching via Stochastic Velocity Field Sampling
• Efficient Molecular Conformer Generation with SO(3)-Averaged Flow Matching and Reflow
• FreqPolicy: Efficient Flow-based Visuomotor Policy via Frequency Consistency
• StraightDP: Geometry-Aware Differential Privacy for Rectified-Flow Transformers
• Evo-0: Vision-Language-Action Model with Implicit Spatial Understanding
• Investigating Stochastic Methods for Prosody Modeling in Speech Synthesis
• FA-Seg: A Fast and Accurate Diffusion-Based Method for Open-Vocabulary Segmentation
• Flow-Modulated Scoring for Semantic-Aware Knowledge Graph Completion
• FunDiff: Diffusion Models over Function Spaces for Physics-Informed Generative Modeling
• Telegrapher's Generative Model via Kac Flows
• VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Models
• FlowRAM: Grounding Flow Matching Policy with Region-Aware Mamba Framework for Robotic Manipulation
• Improving Rectified Flow with Boundary Conditions
• Time-dependent density estimation using binary classifiers
• FlowOE: Imitation Learning with Flow Policy from Ensemble RL Experts for Optimal Execution under Heston Volatility and Concave Market Impacts
• Rectified Point Flow: Generic Point Cloud Pose Estimation
• On Fitting Flow Models with Large Sinkhorn Couplings
• SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
• Efficient Regression-Based Training of Normalizing Flows for Boltzmann Generators
• Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better
• Optimization-Free Diffusion Model -- A Perturbation Theory Approach
• ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation
• On the Relation between Rectified Flows and Optimal Transport
• QWRF-Net: A Quantum-Wavelet Framework with Rectified Flow for Short-Term Precipitation Nowcasting
• Forward-only Diffusion Probabilistic Models
• MEgoHand: Multimodal Egocentric Hand-Object Interaction Motion Generation
• CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learning
• DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Driving
• Open-Set Visual Text Forensics via Sparse-Constraint Rectified Flow
• Minimum-Excess-Work Guidance
• Few-Step Diffusion via Score identity Distillation
• OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoning
• SARe: Structure-Aware Generative 3D Fragment Reassembly
• Flow Sampling: Learning to Sample from Unnormalized Densities via Denoising Conditional Processes
• OmniVLA-RL: A Vision-Language-Action Model with Spatial Understanding and Online RL
• π_0.5: a Vision-Language-Action Model with Open-World Generalization
• MusFlow: Multimodal Music Generation via Conditional Flow Matching

Discussions

• Iterated RF with conservative vector fields should get to OT, though training remains a challenge arxiv.org/abs/2209.14577 [bsky, 2 points, 1 comments]

Related

• On the Relation between Rectified Flows and Optimal Transport
• Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
• Optimal transport mapping via input convex neural networks
• Optimal transport problems regularized by generic convex functions: A geometric and algorithmic approach
• Distorted optimal transport