ChemOrch: Empowering LLMs with Chemical Intelligence via Synthetic Instructions

2025/09/20 by Yi Huang, Huang, Yue, Jiang, Zhengzhe +26 · 2 citations
Computer Science · Decision Sciences · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Scientific Computing and Data Management #Semantic Web and Ontologies

paper · pdf · doi:10.48550/arxiv.2509.16543

openalex publication_date 2025/09/20 · openalex created_date 2025/10/16 · openalex updated_date 2026/07/28

Abstract

Empowering large language models (LLMs) with chemical intelligence remains a challenge due to the scarcity of high-quality, domain-specific instruction-response datasets and the misalignment of existing synthetic data generation pipelines with the inherently hierarchical and rule-governed structure of chemical information. To address this, we propose ChemOrch, a framework that synthesizes chemically grounded instruction-response pairs through a two-stage process: task-controlled instruction generation and tool-aware response construction. ChemOrch enables controllable diversity and levels of difficulty for the generated tasks, and ensures response precision through tool planning and distillation, and tool-based self-repair mechanisms. The effectiveness of ChemOrch is evaluated based on: 1) the high quality of generated instruction data, demonstrating superior diversity and strong alignment with chemical constraints; 2) the reliable generation of evaluation tasks that more effectively reveal LLM weaknesses in chemistry; and 3) the significant improvement of LLM chemistry capabilities when the generated instruction data are used for fine-tuning. Our work thus represents a critical step toward scalable and verifiable chemical intelligence in LLMs.

Citations

• MetaSynth: Meta-Prompting-Driven Agentic Scaffolds for Diverse Synthetic Data Generation
• ELTEX: A Framework for Domain-Driven Synthetic Data Generation
• Oasis: One Image is All You Need for Multimodal Instruction Data Synthesis
• ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning
• ASKCOS: an open source software suite for synthesis planning
• MetaScientist: A Human-AI Synergistic Framework for Automated Mechanical Metamaterial Design
• Phi-4 Technical Report
• PubChem 2025 update
• AutoBench-V: Can Large Vision-Language Models Benchmark Themselves?
• Multimodal Large Language Models for Inverse Molecular Design with Retrosynthetic Planning
• Can LLMs Generate Diverse Molecules? Towards Alignment with Structural Diversity
• HoneyComb: A Flexible LLM-Based Agent System for Materials Science
• The Llama 3 Herd of Models
• Scikit-fingerprints: easy and efficient computation of molecular fingerprints in Python
• DataGen: Unified Synthetic Dataset Generation via Large Language Models
• A Review of Large Language Models and Autonomous Agents in Chemistry
• MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction
• A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery
• SyntheT2C: Generating Synthetic Data for Fine-Tuning Large Language Models on the Text2Cypher Task
• Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing
• MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
• Aligning to Thousands of Preferences via System Message Generalization
• The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
• ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence
• GP-MoLFormer: A Foundation Model For Molecular Generation
• LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
• Dynamic Evaluation of Large Language Models by Meta Probing Agents
• ChemLLM: A Chemical Large Language Model
• Understanding the planning of LLM agents: A survey
• Large Language Model based Multi-Agents: A Survey of Progress and Challenges
• Autonomous chemical research with large language models
• Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations
• Simple synthetic data reduces sycophancy in large language models
• SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models
• ChatDev: Communicative Agents for Software Development
• Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks
• ChemCrow: Augmenting large-language models with chemistry tools
• AugGPT: Leveraging ChatGPT for Text Data Augmentation
• Generating Datasets with Pretrained Language Models
• Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development
• Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question\n Answering
• Interleaved Scene Graphs for Interleaved Text-and-Image Generation Assessment

Cited by

• Synthetic Interaction Data for Scalable Personalization in Large Language Models
• SkillGen: Verified Inference-Time Agent Skill Synthesis

Related

• Developing ChemDFM as a large language foundation model for chemistry
• LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset
• CheMatAgent: Enhancing LLMs for Chemistry and Materials Science through Tree-Search Based Tool Learning
• ChemEval: A Comprehensive Multi-Level Chemical Evaluation for Large Language Models
• ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning