From Query to Logic: Ontology-Driven Multi-Hop Reasoning in LLMs

2025/08/02 by Bian, Haonan, Qi, Yutao, Yang, Rui +4
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2508.01424

Abstract

Large Language Models (LLMs), despite their success in question answering, exhibit limitations in complex multi-hop question answering (MQA) tasks that necessitate non-linear, structured reasoning. This limitation stems from their inability to adequately capture deep conceptual relationships between entities. To overcome this challenge, we present **ORACLE** (**O**ntology-driven **R**easoning **A**nd **C**hain for **L**ogical **E**ucidation), a training-free framework that combines LLMs' generative capabilities with the structural benefits of knowledge graphs. Our approach operates through three stages: (1) dynamic construction of question-specific knowledge ontologies using LLMs, (2) transformation of these ontologies into First-Order Logic reasoning chains, and (3) systematic decomposition of the original query into logically coherent sub-questions. Experimental results on several standard MQA benchmarks show that our framework achieves highly competitive performance, rivaling current state-of-the-art models like DeepSeek-R1. Detailed analyses further confirm the effectiveness of each component, while demonstrating that our method generates more logical and interpretable reasoning chains than existing approaches.

Citations

• Credible Plan-Driven RAG Method for Multi-Hop Question Answering
• MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs
• Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus
• Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs
• Faithful Logical Reasoning via Symbolic Chain-of-Thought
• ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent
• Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph
• Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy
• Generating Data for Symbolic Language with Large Language Models
• Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning
• Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
• Measuring and Narrowing the Compositionality Gap in Language Models
• Distributing Accountability, Not Capability: Phase Separation and the LLM Workflow Quadrant in Autonomous AI Agent Architectures
• Training language models to follow instructions with human feedback
• MuSiQue: Multihop Questions via Single-hop Question Composition
• Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
• Language Models are Few-Shot Learners
• Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings
• Hierarchical Graph Network for Multi-hop Question Answering
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering

Related

• Logic Query of Thoughts: Guiding Large Language Models to Answer Complex Logic Queries with Knowledge Graphs
• Efficient Multi-Hop Question Answering over Knowledge Graphs via LLM Planning and Embedding-Guided Search
• KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration
• Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning
• HOLMES: Hyper-Relational Knowledge Graphs for Multi-hop Question Answering using LLMs