System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcuts

2025/05/25 by Xiaoqiang Wang, Wang, Xiaoqiang, Suyuchen Wang +5 · 6 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2505.18962

openalex publication_date 2025/05/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Chain-of-thought (CoT) reasoning enables large language models (LLMs) to move beyond fast System-1 responses and engage in deliberative System-2 reasoning. However, this comes at the cost of significant inefficiency due to verbose intermediate output. Recent latent-space reasoning methods improve efficiency by operating on hidden states without decoding into language, yet they treat all steps uniformly, failing to distinguish critical deductions from auxiliary steps and resulting in suboptimal use of computational resources. In this paper, we propose System-1.5 Reasoning, an adaptive reasoning framework that dynamically allocates computation across reasoning steps through shortcut paths in latent space. Specifically, System-1.5 Reasoning introduces two types of dynamic shortcuts. The model depth shortcut (DS) adaptively reasons along the vertical depth by early exiting non-critical tokens through lightweight adapter branches, while allowing critical tokens to continue through deeper Transformer layers. The step shortcut (SS) reuses hidden states across the decoding steps to skip trivial steps and reason horizontally in latent space. Training System-1.5 Reasoning involves a two-stage self-distillation process: first distilling natural language CoT into latent-space continuous thought, and then distilling full-path System-2 latent reasoning into adaptive shortcut paths (System-1.5 Reasoning). Experiments on reasoning tasks demonstrate the superior performance of our method. For example, on GSM8K, System-1.5 Reasoning achieves reasoning performance comparable to traditional CoT fine-tuning methods while accelerating inference by over 20x and reducing token generation by 92.31% on average.

Citations

• Seeing Beyond Words: MatVQA for Challenging Visual-Scientific Reasoning in Materials Science
• Phi-4-reasoning Technical Report
• Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems
• A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
• Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation
• Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
• How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach
• CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation
• Chain of Draft: Thinking Faster by Writing Less
• From System 1 to System 2: A Survey of Reasoning Large Language Models
• R³Mem: Bridging Memory Retention and Retrieval via Reversible Compression
• SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs
• FastMCTS: A Simple Sampling Strategy for Data Synthesis
• Atom of Thoughts for Markov LLM Test-Time Scaling
• Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning
• s1: Simple test-time scaling
• O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning
• Kimi k1.5: Scaling Reinforcement Learning with LLMs
• UI-TARS: Pioneering Automated GUI Interaction with Native Agents
• PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
• Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
• Compressed Chain of Thought: Efficient Reasoning Through Dense Representations
• Reasoning Beyond Words ? Exploring framework for hidden state reasoning
• Can Language Models Learn to Skip Steps?
• Fast Best-of-N Decoding via Speculative Rejection
• GPT-4o System Card
• OSCAR: Operating System Control via State-Aware Reasoning and Re-Planning
• Visual Agents as Fast and Slow Thinkers
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
• The Llama 3 Herd of Models
• System-1.x: Learning to Balance Fast and Slow Planning with Language Models
• Distilling System 2 into System 1
• From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step
• Octo: An Open-Source Generalist Robot Policy
• Let's Think Dot by Dot: Hidden Computation in Transformer Language Models
• Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
• LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
• Mixtral of Experts
• Towards Learning a Generalist Model for Embodied Navigation
• Implicit Chain of Thought Reasoning via Knowledge Distillation
• SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
• Think before you speak: Training Language Models With Pause Tokens
• MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inference
• Voyager: An Open-Ended Embodied Agent with Large Language Models
• CoLT5: Faster Long-Range Transformers with Conditional Computation
• PaLM-E: An Embodied Multimodal Language Model
• Fast Inference from Transformers via Speculative Decoding
• Confident Adaptive Language Modeling
• Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
• Training language models to follow instructions with human feedback
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Training Verifiers to Solve Math Word Problems
• LoRA: Low-Rank Adaptation of Large Language Models
• Measuring Mathematical Problem Solving With the MATH Dataset
• Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
• CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade
• DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Depth-Adaptive Transformer
• A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?
• OpenAI o1 System Card

Cited by

• KaVa: Latent Reasoning via Compressed KV-Cache Distillation
• SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMs
• Learning to Ponder: Adaptive Reasoning in Latent Space
• Implicit Reasoning in Large Language Models: A Comprehensive Survey
• Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey
• A Survey on Latent Reasoning

Related

• Reasoning Beyond Words ? Exploring framework for hidden state reasoning
• Break the Chain: Large Language Models Can be Shortcut Reasoners
• Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts
• Adaptive Deep Reasoning: Triggering Deep Thinking When Needed
• Self-Training Elicits Concise Reasoning in Large Language Models