PiERN: Token-Level Routing for Integrating High-Precision Computation and Reasoning
2025/09/17 by Xiao, Hengbo, Jingyuan Fan, Xin Tong +8
Computer Science · #Computation and Language (cs.CL) #Computational Engineering #Embedded Systems Design Techniques #FOS: Computer and information sciences #Finance #Interconnection Networks and Systems #Machine Learning (cs.LG) #Parallel Computing and Optimization Techniques #and Science (cs.CE)
paper · pdf · doi:10.48550/arxiv.2509.18169
openalex publication_date 2025/09/17 · openalex created_date 2025/10/19 · openalex updated_date 2026/07/28
Abstract
Tasks on complex systems require high-precision numerical computation to support decisions, but current large language models (LLMs) cannot integrate such computations as an intrinsic and interpretable capability with existing architectures. Multi-agent approaches can leverage external experts, but inevitably introduce communication overhead and suffer from inefficiency caused by limited scalability. To this end, we propose Physically-isolated Experts Routing Network (PiERN), an architecture for integrating computation and reasoning. Instead of the tool-use workflows or function-calling, PiERN endogenously integrates computational capabilities into neural networks after separately training experts, a text-to-computation module, and a router. At inference, the router directs computation and reasoning at the token level, thereby enabling iterative alternation within a single chain of thought. We evaluate PiERN on representative linear and nonlinear computation-reasoning tasks against LLM finetuning and the multi-agent system approaches. Results show that the PiERN architecture achieves not only higher accuracy than directly finetuning LLMs but also significant improvements in response latency, token usage, and GPU energy consumption compared with mainstream multi-agent approaches. PiERN offers an efficient, interpretable, and scalable paradigm for interfacing language models with scientific systems.
Citations
- AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework
- Text-Trained LLMs Can Zero-Shot Extrapolate PDE Dynamics, Revealing a Three-Stage In-Context Learning Mechanism
- WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent
- A Survey on Large Language Models for Mathematical Reasoning
- Scaling Laws for Native Multimodal Models
- A Survey on Feedback-based Multi-step Reasoning for Large Language Models on Mathematics
- Search-o1: Agentic Search-Enhanced Large Reasoning Models
- Probing the limitations of multimodal language models for chemistry and materials research
- Evaluating World Models with LLM for Decision Making
- Number Cookbook: Number Understanding of Language Models and How to Improve It
- GPT-4o System Card
- Counting Ability of Large Language Models and Impact of Tokenization
- How Numerical Precision Affects Arithmetical Reasoning Capabilities of LLMs
- MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection
- Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System
- Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
- ChartMoE: Mixture of Diversely Aligned Expert Connector for Chart Understanding
- The Llama 3 Herd of Models
- Revisiting Catastrophic Forgetting in Large Language Model Tuning
- Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correction
- Transformers Can Do Arithmetic with the Right Embeddings
- Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts
- CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts
- LLMs can Find Mathematical Reasoning Mistakes by Pedagogical Chain-of-Thought
- LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
- Chronos: Learning the Language of Time Series
- OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
- MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
- Multimodal Large Language Models: A Survey
- A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
- CogVLM: Visual Expert for Pretrained Language Models
- Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE
- LLM4Drive: A Survey of Large Language Models for Autonomous Driving
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
- Understanding Catastrophic Forgetting in Language Models via Implicit Inference
- The first step is the hardest: Pitfalls of Representing and Tokenizing Temporal Data for Large Language Models
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- EVE: Efficient Vision-Language Pre-training with Masked Prediction and Modality-Aware MoE
- AIKernel Semantic DSL Compiler and Deterministic Agent Execution Architecture
- Scientific discovery in the age of artificial intelligence
- Meta-Transformer: A Unified Framework for Multimodal Learning
- Faith and Fate: Limits of Transformers on Compositionality
- Gorilla: Large Language Model Connected with Massive APIs
- A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity
- Evaluating the Performance of Large Language Models on GAOKAO Benchmark
- UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild
- API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
- CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
- GPT-4 Technical Report
- Toolformer: Language Models Can Teach Themselves to Use Tools
- PDEBENCH: An Extensive Benchmark for Scientific Machine Learning
- Distributing Accountability, Not Capability: Phase Separation and the LLM Workflow Quadrant in Autonomous AI Agent Architectures
- Limitations of Language Models in Arithmetic and Symbolic Induction
- BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
- Learning Transferable Visual Models From Natural Language Supervision
- Open Problems in Cooperative AI
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Measuring Massive Multitask Language Understanding
- Representation Learning with Contrastive Predictive Coding
- GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
- Adaptive Mixtures of Local Experts
- HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Related