Large Language Models Are Neurosymbolic Reasoners

2024/01/17 by Meng Fang, Shilong Deng, Fang, Meng +12 · 1 voice · 11 citations
Computer Science · Psychology · #Action (physics) #Artificial intelligence #Computer science #Focus (optics) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Programming language #Psychology #Set (abstract data type) #Sorting #Symbolic execution #The Symbolic #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2401.09334

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2024/01/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

A wide range of real-world applications is characterized by their symbolic nature, necessitating a strong capability for symbolic reasoning. This paper investigates the potential application of Large Language Models (LLMs) as symbolic reasoners. We focus on text-based games, significant benchmarks for agents with natural language capabilities, particularly in symbolic tasks like math, map reading, sorting, and applying common sense in text-based worlds. To facilitate these agents, we propose an LLM agent designed to tackle symbolic challenges and achieve in-game objectives. We begin by initializing the LLM agent and informing it of its role. The agent then receives observations and a set of valid actions from the text-based games, along with a specific symbolic module. With these inputs, the LLM agent chooses an action and interacts with the game environments. Our experimental results demonstrate that our method significantly enhances the capability of LLMs as automated agents for symbolic reasoning, and our LLM agent is effective in text-based games involving symbolic tasks, achieving an average performance of 88% across all tasks.

Cited by

• Causal-Audit: Explicit and Auditable Graph-based Reasoning via Target-Aware Causal Chain Construction
• NeSyPr: Neurosymbolic Proceduralization For Efficient Embodied Reasoning
• Enabling Fine-Grained Operating Points for Black-Box LLMs
• Benchmarking Foundation Models with Retrieval-Augmented Generation in Olympic-Level Physics Problem Solving
• Neuro-Symbolic Frameworks: Conceptual Characterization and Empirical Comparative Analysis
• Can Large Language Models Adequately Perform Symbolic Reasoning Over Time Series?
• Model-Grounded Symbolic Artificial Intelligence Systems Learning and Reasoning with Model-Grounded Symbolic Artificial Intelligence Systems
• Enhancing Large Language Models with Neurosymbolic Reasoning for Multilingual Tasks
• Answer Convergence as a Signal for Early Stopping in Reasoning
• Incentivizing Strong Reasoning from Weak Supervision
• Monte Carlo Planning with Large Language Model for Text-Based Game Agents

Discussions

• Large Language Models Are Neurosymbolic Reasoners [hn, 107 points, 164 comments]

Related

• Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models
• Behavior Cloned Transformers are Neurosymbolic Reasoners
• Enhance Reasoning for Large Language Models in the Game Werewolf
• Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Models
• LOA: Logical Optimal Actions for Text-based Interaction Games