Learning inverse folding from millions of predicted structures

2022/04/10 by Chloe Hsu, Robert Verkuil, Jason Liu +5 · 1 voice · 74 citations
Biochemistry, Genetics and Molecular Biology · Materials Science · #Enzyme Structure and Function #Protein Structure and Dynamics #RNA and protein synthesis mechanisms

paper · pdf · doi:10.1101/2022.04.10.487779

openalex publication_date 2022/04/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30

Abstract

Abstract We consider the problem of predicting a protein sequence from its backbone atom coordinates. Machine learning approaches to this problem to date have been limited by the number of available experimentally determined protein structures. We augment training data by nearly three orders of magnitude by predicting structures for 12M protein sequences using AlphaFold2. Trained with this additional data, a sequence-to-sequence transformer with invariant geometric input processing layers achieves 51% native sequence recovery on structurally held-out backbones with 72% recovery for buried residues, an overall improvement of almost 10 percentage points over existing methods. The model generalizes to a variety of more complex tasks including design of protein complexes, partially masked structures, binding interfaces, and multiple states.

Citations

• Improved protein structure prediction using predicted interresidue orientations
• Improved protein structure prediction using potentials from deep learning
• Structure of the SARS-CoV-2 spike receptor-binding domain bound to the ACE2 receptor
• Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences
• ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
• Highly accurate protein structure prediction with AlphaFold
• Accurate prediction of protein structures and interactions using a three-track neural network
• КОМПЬЮТЕРНАЯ ТОМОГРАФИЯ – ОСНОВА ОПТИМИЗАЦИИ МИНИТОРАКОТОМНОГО ДОСТУПА

Cited by

• InstructPLM-mu: 1-Hour Fine-Tuning of ESM2 Beats ESM3 in Protein Mutation Predictions
• Evolutionary-scale prediction of atomic-level protein structure with a language model
• Limitations of the refolding pipeline for de novo protein design
• AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation
• Generalizable and scalable protein stability prediction with rewired protein generative models
• Learning the shape of protein microenvironments with a holographic convolutional neural network
• Predicting absolute protein folding stability using generative models
• SSEmb: A joint embedding of protein sequence and structure enables robust variant effect predictions
• Are protein language models the new universal key?
• Simulating 500 million years of evolution with a language model
• Protein Language Models: Is Scaling Necessary?
• Kernel-Based Evaluation of Conditional Biological Sequence Models
• PRISM: Enhancing Protein Inverse Folding through Fine-Grained Retrieval on Structure-Sequence Multimodal Representations
• Re‐engineering of a carotenoid‐binding protein based on NMR structure
• Evolutionary Profiles for Protein Fitness Prediction
• SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation
• Flow Autoencoders are Effective Protein Tokenizers
• ABConformer: Physics-inspired Sliding Attention for Antibody-Antigen Interface Prediction
• Context-aware geometric deep learning for protein sequence design
• SpecMER: Fast Protein Generation with K-mer Guided Speculative Decoding
• Decoding molecular mechanisms for loss-of-function variants in the human proteome
• Annotation Vocabulary (Might Be) All You Need
• Contextual protein and antibody encodings from equivariant graph transformers
• Understanding language model scaling for protein fitness prediction
• Tools For Building Artificial Biological Nanostructures
• Protein engineering in the deep learning era
• RemoteFoldSet: Benchmarking Structural Awareness of Protein Language Models
• Robust deep learning–based protein sequence design using ProteinMPNN
• De novo Design of All-atom Biomolecular Interactions with RFdiffusion3
• ShortListing Model: A Streamlined SimplexDiffusion for Discrete Variable Generation
• Tokenizing Loops of Antibodies
• Supervised learning of protein variant effects across large‐scale mutagenesis datasets
• BConformeR: A Conformer Based on Mutual Sampling for Unified Prediction of Continuous and Discontinuous Antibody Binding Sites
• Chem3DLLM: 3D Multimodal Large Language Models for Chemistry
• Artificial intelligence-driven computational methods for antibody design and optimization
• Multi-state Protein Design with DynamicMPNN
• Learning from B Cell Evolution: Adaptive Multi-Expert Diffusion for Antibody Design via Online Optimization
• Comparative evaluation of methods for the prediction of protein–ligand binding sites
• AlphaFold Database Debiasing for Robust Inverse Folding
• AMix-1: A Pathway to Test-Time Scalable Protein Foundation Model
• Predicting mutational effects on protein binding from folding energy
• Discrete Diffusion Trajectory Alignment via Stepwise Decomposition
• All-atom inverse protein folding through discrete flow matching
• HelixDesign-Antibody: A Scalable Production-Grade Platform for Antibody Design Built on HelixFold3
• Do Protein Transformers Have Biological Intelligence?
• Watermarking Autoregressive Image Generation
• ProteinZero: Self-Improving Protein Generation via Online Reinforcement Learning
• EnerBridge-DPO: Energy-Guided Protein Inverse Folding with Markov Bridges and Direct Preference Optimization
• Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approach
• Protein Inverse Folding From Structure Feedback
• ProtInvTree: Deliberate Protein Inverse Folding with Reward-guided Tree Search
• Improving Protein Sequence Design through Designability Preference Optimization
• ProSpero: Active Learning for Robust Protein Design Beyond Wild-Type Neighborhoods
• CFP-Gen: Combinatorial Functional Protein Generation via Diffusion Language Models
• HelixDesign-Binder: A Scalable Production-Grade Platform for Binder Design Built on HelixFold3
• Designing Cyclic Peptides via Harmonic SDE with Atom-Bond Modeling
• PDFBench: A Benchmark for De novo Protein Design from Function
• AbBiBench: A Benchmark for Antibody Binding Affinity Maturation and Design
• Machine learning for evolutionary-based and physics-inspired protein design: Current and future synergies
• VARD: Efficient and Dense Fine-Tuning for Diffusion Models with Value-based RL
• VenusX: Unlocking Fine-Grained Functional Understanding of Proteins
• Democratizing Protein Language Model Training, Sharing and Collaboration
• InstructPLM: Aligning Protein Language Models to Follow Protein Structure Instructions
• Sequence-Only Prediction of Binding Affinity Changes: A Robust and Interpretable Model for Antibody Engineering
• Structure of the space of folding protein sequences defined by large language models
• UMA-Inverse: Ligand-Conditioned Protein Inverse Folding with a Distogram-Supervised Dense Pair Encoder
• AlloGen: Conformation-Selective Binder Generation with Differential State Scoring
• Demystifying Multimodal Biomolecular Co-design With Intrinsic Geodesic Coupling
• SurfDesign: Effective Protein Design on Molecular Surfaces
• Yeti: A compact protein structure tokenizer for reconstruction and multi-modal generation
• CAPSUL: A Comprehensive Human Protein Benchmark for Subcellular Localization
• EnzyPGM: Pocket-conditioned Generative Model for Substrate-specific Enzyme Design
• Scaffolding protein functional sites using deep learning
• An All-Atom Generative Model for Designing Protein Complexes

Discussions

• Major differences are also observed when using these models as training data. Hsu et al found that inverse folding methods trained on 10^4 to 10^5 experimental structures outperformed those trained on [bsky, 4 points, 1 comments]

Related

• Towards Solving the Inverse Protein Folding Problem
• Inverse Protein Folding Using Deep Bayesian Optimization
• PiFold: Toward effective and efficient protein inverse folding
• From sequence to protein structure and conformational dynamics with artificial intelligence/machine learning
• Highly accurate protein structure prediction with AlphaFold