SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery

2019/11/12 by Shion Honda, Honda, Shion, Shoi Shi +3 · 16 citations
Computer Science · Materials Science · Biochemistry, Genetics and Molecular Biology · #Computational Drug Discovery Methods #Machine Learning in Materials Science #Protein Structure and Dynamics

paper · pdf · doi:10.48550/arxiv.1911.04738

Abstract

In drug-discovery-related tasks such as virtual screening, machine learning is emerging as a promising way to predict molecular properties. Conventionally, molecular fingerprints (numerical representations of molecules) are calculated through rule-based algorithms that map molecules to a sparse discrete space. However, these algorithms perform poorly for shallow prediction models or small datasets. To address this issue, we present SMILES Transformer. Inspired by Transformer and pre-trained language models from natural language processing, SMILES Transformer learns molecular fingerprints through unsupervised pre-training of the sequence-to-sequence language model using a huge corpus of SMILES, a text representation system for molecules. We performed benchmarks on 10 datasets against existing fingerprints and graph-based methods and demonstrated the superiority of the proposed algorithms in small-data settings where pre-training facilitated good generalization. Moreover, we define a novel metric to concurrently measure model accuracy and data efficiency.

Citations

• Auto-Encoding Variational Bayes

Cited by

• MS-GPT: Rethinking MS/MS De Novo Structure Elucidation as Spectrum-Induced Posterior Querying of a Molecule-Language Model
• Circuits, Features, and Heuristics in Molecular Transformers
• Digital Surfactant
• Rep3Net: An Approach Exploiting Multimodal Representation for Molecular Bioactivity Prediction
• A decoupled alignment kernel for peptide membrane permeability predictions
• AssayMatch: Learning to Select Data for Molecular Activity Models
• Learning Attributed Graph Representations with Communicative Message Passing Transformer
• Using Clinical Drug Representations for Improving Mortality and Length of Stay Predictions
• SoDaDE: Solvent Data-Driven Embeddings with Small Transformer Models
• Evolutionary Tree in Chemical Space of Natural Products
• Multimodal Regression for Enzyme Turnover Rates Prediction
• Chemical-Reaction-Aware Molecule Representation Learning
• SOLD: SELFIES-based Objective-driven Latent Diffusion
• Artificial Intelligence in Drug Discovery: Applications and Techniques
• Artificial Intelligence and Generative Models for Materials Discovery -- A Review
• Learn molecular representations from large-scale unlabeled molecules for drug discovery

Related

• SmilesT5: Domain-specific pretraining for molecular language models
• Stepping Back to SMILES Transformers for Fast Molecular Representation Inference
• Synergy Effect between Convolutional Neural Networks and the Multiplicity of SMILES for Improvement of Molecular Prediction
• Molecular Fingerprints for Robust and Efficient ML-Driven Molecular Generation
• SMILES-Mamba: Chemical Mamba Foundation Models for Drug ADMET Prediction