π-CoT: Prolog-Initialized Chain-of-Thought Prompting for Multi-Hop Question-Answering

2025/06/25 by Chao Wan, Wan, Chao, Aobo Gong +9
Arts and Humanities · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Narrative Theory and Analysis

paper · pdf · doi:10.48550/arxiv.2506.20642

openalex publication_date 2025/06/25 · openalex created_date 2025/10/09 · openalex updated_date 2026/08/03

Abstract

Chain-of-Thought (CoT) prompting significantly enhances large language models' (LLMs) problem-solving capabilities, but still struggles with complex multi-hop questions, often falling into circular reasoning patterns or deviating from the logical path entirely. This limitation is particularly acute in retrieval-augmented generation (RAG) settings, where obtaining the right context is critical. We introduce Prolog-Initialized Chain-of-Thought (π-CoT), a novel prompting strategy that combines logic programming's structural rigor with language models' flexibility. π-CoT reformulates multi-hop questions into Prolog queries decomposed as single-hop sub-queries. These are resolved sequentially, producing intermediate artifacts, with which we initialize the subsequent CoT reasoning procedure. Extensive experiments demonstrate that π-CoT significantly outperforms standard RAG and in-context CoT on multi-hop question-answering benchmarks.

Citations

• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning
• PhantomWiki: On-Demand Datasets for Reasoning and Retrieval Evaluation
• From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
• Chain-of-Retrieval Augmented Generation
• DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
• Search-o1: Agentic Search-Enhanced Large Reasoning Models
• LightRAG: Simple and Fast Retrieval-Augmented Generation
• HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly
• The Llama 3 Herd of Models
• Thought-Like-Pro: Enhancing Reasoning of Large Language Models through Self-Driven Prolog-based Chain-of-Thought
• Reliable Reasoning Beyond Natural Language
• Arithmetic Reasoning with LLM: Prolog Generation & Permutation
• NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models
• HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models
• From Local to Global: A Graph RAG Approach to Query-Focused Summarization
• RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
• Retrieval-Augmented Generation for Large Language Models: A Survey
• CareerX: A Retrieval-Augmented Generation Framework for Personalized AI-Driven Career Guidance
• Efficient Memory Management for Large Language Model Serving with PagedAttention
• Lost in the Middle: How Language Models Use Long Contexts
• Faith and Fate: Limits of Transformers on Compositionality
• Hierarchical Prompting Assists Large Language Model on Web Navigation
• ViperGPT: Visual Inference via Python Execution for Reasoning
• Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP
• Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
• When to Use What: An In-Depth Comparative Empirical Analysis of OpenIE Systems for Downstream Applications
• Measuring and Narrowing the Compositionality Gap in Language Models
• Automatic Chain of Thought Prompting in Large Language Models
• Distributing Accountability, Not Capability: Phase Separation and the LLM Workflow Quadrant in Autonomous AI Agent Architectures
• Decomposed Prompting: A Modular Approach for Solving Complex Tasks
• Is a Question Decomposition Unit All We Need?
• A Survey on Neural Open Information Extraction: Current Status and Future Directions
• Large Language Models are Zero-Shot Reasoners
• Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• MuSiQue: Multihop Questions via Single-hop Question Composition
• Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Break It Down: A Question Understanding Benchmark
• Compositional Questions Do Not Necessitate Multi-hop Reasoning
• NLProlog: Reasoning with Weak Unification for Question Answering in\n Natural Language
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
• Learning to Map Sentences to Logical Form: Structured Classification\n with Probabilistic Categorial Grammars
• Human problem solving: The state of the theory in 1970.
• Understanding Dataset Design Choices for Multi-hop Reasoning

Related

• Augmenting LLM Reasoning with Dynamic Notes Writing for Complex QA
• Can Memory-Augmented Language Models Generalize on Reasoning-in-a-Haystack Tasks?
• KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues
• Emulating Retrieval Augmented Generation via Prompt Engineering for Enhanced Long Context Comprehension in LLMs
• Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions