BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations

2023/10/11 by Qizhi Pei, Pei, Qizhi, Wei Zhang +13 · 14 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Artificial Intelligence (cs.AI) #Biomedical Text Mining and Ontologies #Biomolecules (q-bio.BM) #Computation and Language (cs.CL) #Computational Drug Discovery Methods #FOS: Biological sciences #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Bioinformatics

paper · pdf · doi:10.48550/arxiv.2310.07276

openalex publication_date 2023/10/11 · openalex created_date 2023/10/13 · openalex updated_date 2026/07/28

Abstract

Recent advancements in biological research leverage the integration of molecules, proteins, and natural language to enhance drug discovery. However, current models exhibit several limitations, such as the generation of invalid molecular SMILES, underutilization of contextual information, and equal treatment of structured and unstructured knowledge. To address these issues, we propose BioT5, a comprehensive pre-training framework that enriches cross-modal integration in biology with chemical knowledge and natural language associations. BioT5 utilizes SELFIES for 100% robust molecular representations and extracts knowledge from the surrounding context of bio-entities in unstructured biological literature. Furthermore, BioT5 distinguishes between structured and unstructured knowledge, leading to more effective utilization of information. After fine-tuning, BioT5 shows superior performance across a wide range of tasks, demonstrating its strong capability of capturing underlying relations and properties of bio-entities. Our code is available at \hrefhttps://github.com/QizhiPei/BioT5Github.

Cited by

• BioMedGPT-Mol: Multi-task Learning for Molecular Understanding and Generation
• OsmT: Bridging OpenStreetMap Queries and Natural Language with Open-source Tag-aware Language Models
• MolEdit: Knowledge Editing for Multimodal Molecule Language Models
• Chain-of-Generation: Progressive Latent Diffusion for Text-Guided Molecular Design
• General Intelligence-based Fragmentation (GIF): A framework for peak-labeled spectra simulation
• Protein as a Second Language for LLMs
• BioVERSE: Representation Alignment of Biomedical Modalities to LLMs for Multi-Modal Reasoning
• Automated extraction of fungal trophic modes from literature using BioBERT: an open pilot workflow
• POLO: Preference-Guided Multi-Turn Reinforcement Learning for Lead Optimization
• OSCAgent: Accelerating the Discovery of Organic Solar Cells with LLM Agents
• Training Text-to-Molecule Models with Context-Aware Tokenization
• MolErr2Fix: Benchmarking LLM Trustworthiness in Chemistry via Modular Error Detection, Localization, Explanation, and Revision
• CROP: Integrating Topological and Spatial Structures via Cross-View Prefixes for Molecular LLMs
• Large Language Models Transform Organic Synthesis From Reaction Prediction to Automation

Related

• BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning
• Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Survey
• Advancing biomolecular understanding and design following human instructions
• BioMedGPT-Mol: Multi-task Learning for Molecular Understanding and Generation
• BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicine