Advancing bioinformatics with language models: components, applications, and perspectives

2024/01/08 by Jiajia Liu, Liu, Jiajia, Mengyuan Yang +12 · 1 voice · 3 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Artificial intelligence #Artificial neural network #Computer science #Data science #Focus (optics) #Foundation (evidence) #Language model #Lexical analysis #Machine Learning in Bioinformatics #Machine learning #Natural language #Natural language processing #RNA modifications and cancer #Single-cell and spatial transcriptomics #Topic Modeling #Transformer

paper · pdf · open access · doi:10.1093/bib/bbag367

published in Briefings in Bioinformatics 27(4) (Oxford University Press)

openalex publication_date 2026/07/01 · openalex created_date 2026/07/11 · openalex updated_date 2026/07/23

Abstract

Large language models (LLMs) are deep learning-based artificial intelligence models that have achieved remarkable success in natural language processing. Typically composed of neural networks with billions of parameters, they are trained on massive unlabeled datasets using self-supervised or semi-supervised learning. Beyond language, LLMs hold immense potential for addressing complex bioinformatics challenges. This review provides a comprehensive overview of transformer-based model applications in genomics, transcriptomics, proteomics, drug discovery, and single-cell analysis. We discuss critical components, including tokenization strategies for diverse biological data, transformer architectures, attention mechanisms, and pretraining approaches. We also survey currently available foundation models and their downstream applications across bioinformatics domains. Finally, we highlight major challenges that remain insufficiently addressed in prior reviews and outline future perspectives and design principles for next-generation biological language models, offering practical guidance for both users and developers.

Citations

• A DNA language model based on multispecies alignment predicts the effects of genome-wide variants
• Transfer learning enables predictions in network biology
• LoRA: Low-Rank Adaptation of Large Language Models
• Energy and Policy Considerations for Deep Learning in NLP
• Attention Is All You Need
• Distilling the Knowledge in a Neural Network
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• Genomic Language Models: Opportunities and Challenges

Cited by

• Large Language Models: A Paradigm Shift for Dementia Diagnosis and Care
• Brain Imaging Foundation Models, Are We There Yet? A Systematic Review of Foundation Models for Brain Imaging and Biomedical Research
• From Genotype to Phenotype: Raman Spectroscopy and Machine Learning for Label-Free Single-Cell Analysis

Discussions

• “Large language models in bioinformatics: applications and perspectives” Summarizes different uses and architectures, particularly highlighting difference between BERT and GPT-type models arxiv.org/ [bsky, 3 points, 0 comments]

Related

• Large language models and their applications in bioinformatics
• Large Language Models in Bioinformatics: A Survey
• Large Language Models for Bioinformatics
• Scientific Large Language Models: A Survey on Biological & Chemical Domains
• An Evaluation of Large Language Models in Bioinformatics Research