MolPILE -- large-scale, diverse dataset for molecular representation learning

2025/09/22 by Jakub Adamczyk, Adamczyk, Jakub, Jakub Poziemski +7
Biochemistry, Genetics and Molecular Biology · Computer Science · Materials Science · #Computational Drug Discovery Methods #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Bioinformatics #Machine Learning in Materials Science

paper · pdf · doi:10.48550/arxiv.2509.18353

openalex publication_date 2025/09/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The size, diversity, and quality of pretraining datasets critically determine the generalization ability of foundation models. Despite their growing importance in chemoinformatics, the effectiveness of molecular representation learning has been hindered by limitations in existing small molecule datasets. To address this gap, we present MolPILE, large-scale, diverse, and rigorously curated collection of 222 million compounds, constructed from 6 large-scale databases using an automated curation pipeline. We present a comprehensive analysis of current pretraining datasets, highlighting considerable shortcomings for training ML models, and demonstrate how retraining existing models on MolPILE yields improvements in generalization performance. This work provides a standardized resource for model training, addressing the pressing need for an ImageNet-like dataset in molecular chemistry.

Citations

• Benchmarking Pretrained Molecular Embedding Models For Molecular Representation Learning
• ToxBench: A Binding Affinity Prediction Benchmark with AB-FEP-Calculated Labels for Human Estrogen Receptor Alpha
• Evaluating machine learning models for predicting pesticide toxicity to honey bees
• Phi-4 Technical Report
• WelQrate: Defining the Gold Standard in Small Molecule Drug Discovery Benchmarking
• ChemFM as a Scaling Law Guided Foundation Model Pre-trained on Informative Chemicals
• Measuring Diversity: Axioms and Challenges
• Scikit-fingerprints: easy and efficient computation of molecular fingerprints in Python
• ApisTox: a new benchmark dataset for the classification of small molecules toxicity on honey bees
• Transformers for molecular property prediction: Lessons learned from the past five years
• DataComp: In search of the next generation of multimodal datasets
• SELFormer: Molecular Representation Learning via SELFIES Language Models
• Organometallic anti-tumor agents: targeting from biomolecules to dynamic bioprocesses
• Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language
• SPICE, A Dataset of Drug-like Molecules and Peptides for Training Machine Learning Potentials
• ChemBERTa-2: Towards Chemical Foundation Models
• How Much Space Has Been Explored? Measuring the Chemical Space Covered by Databases and Machine-Generated Molecules
• Relative Molecule Self-Attention Transformer
• Pre-training Molecular Graph Representation with 3D Geometry
• Chemical-Reaction-Aware Molecule Representation Learning
• Scaling Laws for Neural Machine Translation
• Datasets: A Community Library for Natural Language Processing
• Large-Scale Chemical Language Representations Capture Molecular Structure and Properties
• Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development
• The Pile: An 800GB Dataset of Diverse Text for Language Modeling
• Molecular representation learning with language models and domain-relevant auxiliary tasks
• Scaling Laws for Neural Language Models
• Strategies for Pre-training Graph Neural Networks
• MoleculeNet: A Benchmark for Molecular Machine Learning
• PubChem Substance and Compound databases
• Drug discovery beyond the ‘rule-of-five’
• Property distribution of drug-related chemical databases*

Related

• MolTextNet: A Two-Million Molecule-Text Dataset for Multimodal Molecular Learning
• MolE: a molecular foundation model for drug discovery
• ChemPile: A 250GB Diverse and Curated Dataset for Chemical Foundation Models
• ChemFM as a Scaling Law Guided Foundation Model Pre-trained on Informative Chemicals
• ChemBERTa-2: Towards Chemical Foundation Models