Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design

2024/02/07 by Andrew T. Campbell, Campbell, Andrew, Jason Yim +7 · 50 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Biomedical Text Mining and Ontologies #Cancer-related gene regulation #FOS: Biological sciences #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Quantitative Methods (q-bio.QM) #Topic Modeling

paper · doi:10.48550/arxiv.2402.04997

openalex publication_date 2024/02/07 · openalex created_date 2024/02/09 · openalex updated_date 2026/07/28

Abstract

Combining discrete and continuous data is an important capability for generative models. We present Discrete Flow Models (DFMs), a new flow-based model of discrete data that provides the missing link in enabling flow-based generative models to be applied to multimodal continuous and discrete data problems. Our key insight is that the discrete equivalent of continuous space flow matching can be realized using Continuous Time Markov Chains. DFMs benefit from a simple derivation that includes discrete diffusion models as a specific instance while allowing improved performance over existing diffusion-based approaches. We utilize our DFMs method to build a multimodal flow-based modeling framework. We apply this capability to the task of protein co-design, wherein we learn a model for jointly generating protein structure and sequence. Our approach achieves state-of-the-art co-design performance while allowing the same multimodal model to be used for flexible generation of the sequence or structure.

Cited by

• Chamaileon: Cross-Context Binder Design with Contextualized Modeling and Mixed Sampling
• PairFlow: Closed-Form Source-Target Coupling for Few-Step Generation in Discrete Flow Models
• HD-Prot: A Protein Language Model for Joint Sequence-Structure Modeling with Continuous Structure Tokens
• MolGuidance: Advanced Guidance Strategies for Conditional Molecular Generation with Flow Matching
• OMTRA: A Multi-Task Generative Model for Structure-Based Drug Design
• Consistent Synthetic Sequences Unlock Structural Diversity in Fully Atomistic De Novo Protein Design
• Masked Diffusion Models are Secretly Learned-Order Autoregressive Models
• FlexiFlow: decomposable flow matching for generation of flexible molecular ensemble
• Departures: Distributional Transport for Single-Cell Perturbation Prediction with Neural Schrödinger Bridges
• Collaborative Multi-Robot Non-Prehensile Manipulation via Flow-Matching Co-Generation
• VEDA: 3D Molecular Generation via Variance-Exploding Diffusion with Annealing
• Simplex-to-Euclidean Bijections for Categorical Flow Matching
• EnzyControl: Adding Functional and Substrate-Specific Control for Enzyme Backbone Generation
• ODesign: A World Model for Biomolecular Interaction Design
• Latent-Augmented Discrete Diffusion Models
• FLOWR.root: A flow matching based foundation model for joint multi-purpose structure-aware 3D ligand generation and affinity prediction
• Efficient Inference for Coupled Hidden Markov Models in Continuous Time and Discrete Space
• PRISM: Enhancing Protein Inverse Folding through Fine-Grained Retrieval on Structure-Sequence Multimodal Representations
• ProteinAE: Protein Diffusion Autoencoders for Structure Encoding
• Matching the Optimal Denoiser in Point Cloud Diffusion with (Improved) Rotational Alignment
• SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models
• Guided Star-Shaped Masked Diffusion
• Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
• Edit-Based Flow Matching for Temporal Point Processes
• Fine-Tuning Masked Diffusion for Provable Self-Correction
• Drax: Speech Recognition with Discrete Flow Matching
• Rainbow Padding: Mitigating Early Termination in Instruction-Tuned Diffusion LLMs
• OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flows
• Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner
• AReUReDi: Annealed Rectified Updates for Refining Discrete Flows with Multi-Objective Guidance
• Refine Drugs, Don't Complete Them: Uniform-Source Discrete Flows for Fragment-Based Drug Discovery
• Fading to Grow: Growing Preference Ratios via Preference Fading Discrete Diffusion for Recommendation
• Let Physics Guide Your Protein Flows: Topology-aware Unfolding and Generation
• Planner Aware Path Learning in Diffusion Language Models Training
• Entering the Era of Discrete Diffusion Models: A Benchmark for Schrödinger Bridges and Entropic Optimal Transport
• Discrete Guidance Matching: Exact Guidance for Discrete Flow Matching
• Error Analysis of Discrete Flow with Generator Matching
• FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models
• Learnable Sampler Distillation for Discrete Diffusion Models
• Latent-Kernel Discrete Flow Maps for Few-Step Generation
• Exploring Synthesizable Chemical Space with Iterative Pathway Refinements
• Discrete Diffusion Models: Novel Analysis and New Sampler Guarantees
• Guided Sequence-Structure Generative Modeling for Iterative Antibody Optimization
• Monte Carlo Tree Diffusion with Multiple Experts for Protein Design
• DiFlow-TTS: Discrete Flow Matching with Factorized Speech Tokens for Low-Latency Zero-Shot Text-To-Speech
• SafeProtein: Red-Teaming Framework and Benchmark for Protein Foundation Models
• Multimodal Generative Flows for LHC Jets
• Any-Order Flexible Length Masked Diffusion
• Energy-Based Flow Matching for Generating 3D Molecular Structure
• FlowMol3: Flow Matching for 3D De Novo Small-Molecule Generation

Related

• α-Flow: A Unified Framework for Continuous-State Discrete Flow Matching Models
• Branching Flows: Discrete, Continuous, and Manifold Flow Matching with Splits and Deletions
• Unlocking Guidance for Discrete State-Space Diffusion and Flow Models
• Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspective
• Fisher Flow Matching for Generative Modeling over Discrete Data