Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models

2023/06/12 by Soochan Lee, Gunhee Kim, Lee, Soochan +1 · 7 citations
Computer Science · Decision Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Data Quality and Management #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2306.06891

openalex publication_date 2023/06/12 · openalex created_date 2023/06/14 · openalex updated_date 2026/07/28

Abstract

Generating intermediate steps, or Chain of Thought (CoT), is an effective way to significantly improve language models' (LM) multi-step reasoning capability. However, the CoT lengths can grow rapidly with the problem complexity, easily exceeding the maximum context size. Instead of increasing the context limit, which has already been heavily investigated, we explore an orthogonal direction: making LMs divide a problem into multiple contexts. We propose a new inference framework, called Recursion of Thought (RoT), which introduces several special tokens that the models can output to trigger context-related operations. Extensive experiments with multiple architectures including GPT-3 show that RoT dramatically improves LMs' inference capability to solve problems, whose solution consists of hundreds of thousands of tokens.

Cited by

• Understanding Tone-Dependent Inference Cost in Large Language Models
• Serve Programs, Not Prompts
• Pie: A Programmable Serving System for Emerging LLM Applications
• Hybrid Deep Searcher: Integrating Parallel and Sequential Search Reasoning
• ROVER: Recursive Reasoning Over Videos with Vision-Language Models for Embodied Tasks
• Recursive Models for Long-Horizon Reasoning
• bridging ai skills gaps in marketing education: prompt engineering as a key competency

Related

• Distilling Reasoning Capabilities into Smaller Language Models
• Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thought
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models
• Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
• SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought