Foundation Models for AI-Enabled Biological Design

2025/05/16 by Moldwin, Asher, Shehu, Amarda
#Artificial Intelligence (cs.AI) #Biomolecules (q-bio.BM) #FOS: Biological sciences #FOS: Computer and information sciences #Genomics (q-bio.GN) #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2505.11610

Abstract

This paper surveys foundation models for AI-enabled biological design, focusing on recent developments in applying large-scale, self-supervised models to tasks such as protein engineering, small molecule design, and genomic sequence design. Though this domain is evolving rapidly, this survey presents and discusses a taxonomy of current models and methods. The focus is on challenges and solutions in adapting these models for biological applications, including biological sequence modeling architectures, controllability in generation, and multi-modal integration. The survey concludes with a discussion of open problems and future directions, offering concrete next-steps to improve the quality of biological sequence generation.

Citations

• ProtMamba: a homology-aware but alignment-free protein state space model
• Advancing biomolecular understanding and design following human instructions
• Protein-Mamba: Biological Mamba Models for Protein Function Prediction
• Genomic Language Models: Opportunities and Challenges
• DiscDiff: Latent Diffusion Model for DNA Sequence Generation
• It’s time to admit that genes are not the blueprint for life
• Towards Joint Sequence-Structure Generation of Nucleic Acid and Protein Complexes with SE(3)-Discrete Diffusion
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter
• HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution
• Large language models generate functional protein sequences across diverse families
• Hungry Hungry Hippos: Towards Language Modeling with State Space Models
• Efficient Training of Language Models to Fill in the Middle
• ProGen2: Exploring the Boundaries of Protein Language Models
• Diffusion-LM Improves Controllable Text Generation
• Efficiently Modeling Long Sequences with Structured State Spaces
• Autoregressive Diffusion Models
• On the Opportunities and Risks of Foundation Models
• Highly accurate protein structure prediction with AlphaFold
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• Zero-Shot Text-to-Image Generation
• Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions
• Language Models are Few-Shot Learners
• Scaling Laws for Neural Language Models
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Evaluating Protein Transfer Learning with TAPE
• XLNet: Generalized Autoregressive Pretraining for Language Understanding
• Attention Is All You Need
• Gene Ontology: tool for the unification of biology

Related

• Generative AI for Controllable Protein Sequence Design: A Survey
• Progress and Opportunities of Foundation Models in Bioinformatics
• Toward AI-Driven Digital Organism: Multiscale Foundation Models for Predicting, Simulating and Programming Biology at All Levels
• Foundation models in bioinformatics
• Automated Biodesign Engineering by Abductive Meta-Interpretive Learning