Reasoning with Language Model Prompting: A Survey

2022/12/19 by Shuofei Qiao, Qiao, Shuofei, Yixin Ou +15 · 57 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Natural Language Processing Techniques #Semantic Web and Ontologies #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2212.09597

openalex publication_date 2022/12/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Reasoning, as an essential ability for complex problem-solving, can provide back-end support for various real-world applications, such as medical diagnosis, negotiation, etc. This paper provides a comprehensive survey of cutting-edge research on reasoning with language model prompting. We introduce research works with comparisons and summaries and provide systematic resources to help beginners. We also discuss the potential reasons for emerging such reasoning abilities and highlight future research directions. Resources are available at https://github.com/zjunlp/Prompt4ReasoningPapers (updated periodically).

Cited by

• Beyond Correctness: Exposing LLM-generated Logical Flaws in Reasoning via Multi-step Automated Theorem Proving
• Collaborative Causal Sensemaking: Closing the Complementarity Gap in Human-AI Decision Support
• From Hypothesis to Premises: LLM-based Backward Logical Reasoning with Selective Symbolic Translation
• Cross-Lingual Prompt Steerability: Towards Accurate and Robust LLM Behavior across Languages
• DETAIL Matters: Measuring the Impact of Prompt Specificity on Reasoning in Large Language Models
• Structured Definitions and Segmentations for Legal Reasoning in LLMs: A Study on Indian Legal Data
• SPAN: Benchmarking and Improving Cross-Calendar Temporal Reasoning of Large Language Models
• Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learning
• The Riddle of Reflection: Evaluating Reasoning and Self-Awareness in Multilingual LLMs using Indian Riddles
• SoK: Honeypots & LLMs, More Than the Sum of Their Parts?
• Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis
• Reasoning About Reasoning: Towards Informed and Reflective Use of LLM Reasoning in HCI
• Select-Then-Decompose: From Empirical Analysis to Adaptive Selection Strategy for Task Decomposition in Large Language Models
• Merlin's Whisper: Enabling Efficient Reasoning in LLMs via Black-box Adversarial Prompting
• Test-Time Reasoners Are Strategic Multiple-Choice Test-Takers
• Mission Impossible: Feedback-Guided Dynamic Interactive Planning for Improving Reasoning on LLMs
• A Survey on Automatic Credibility Assessment Using Textual Credibility Signals in the Era of Large Language Models
• Scaling Generalist Data-Analytic Agents
• Transformer Tafsir at QIAS 2025 Shared Task: Hybrid Retrieval-Augmented Generation for Islamic Knowledge Question Answering
• PRIME: Planning and Retrieval-Integrated Memory for Enhanced Reasoning
• Causal Consistency Regularization: Training Verifiably Sensitive Reasoning in Large Language Models
• Memp: Exploring Agent Procedural Memory
• Streamlining Admission with LOR Insights: AI-Based Leadership Assessment in Online Master's Program
• Can Large Language Models Generate Effective Datasets for Emotion Recognition in Conversations?
• OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use
• Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning
• Is Large Language Model Performance on Reasoning Tasks Impacted by Different Ways Questions Are Asked?
• Promptomatix: An Automatic Prompt Optimization Framework for Large Language Models
• Agent-Based Detection and Resolution of Incompleteness and Ambiguity in Interactions with Large Language Models
• The Thin Line Between Comprehension and Persuasion in LLMs
• E-FreeM2: Efficient Training-Free Multi-Scale and Cross-Modal News Verification via MLLMs
• Bayesian Social Deduction with Graph-Informed Language Models
• Code Reasoning for Software Engineering Tasks: A Survey and A Call to Action
• AutoMind: Adaptive Knowledgeable Agent for Automated Data Science
• Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledge
• Towards conversational assistants for health applications: using ChatGPT to generate conversations about heart failure
• Safer Prompts: Reducing Risks from Memorization in Visual Generative AI
• When Language Shapes Thought: Cross-Lingual Transfer of Factual Knowledge in Question Answering
• Knowledge Augmented Complex Problem Solving with Large Language Models: A Survey
• ExAnte: A Benchmark for Ex-Ante Inference in Large Language Models
• Iterative Self-Incentivization Empowers Large Language Models as Agentic Searchers
• Which Data Attributes Stimulate Math and Code Reasoning? An Investigation via Influence Functions
• Causal-Invariant Cross-Domain Out-of-Distribution Recommendation
• Veracity Bias and Beyond: Uncovering LLMs' Hidden Beliefs in Problem-Solving Reasoning
• ThinkLess: A Training-Free Inference-Efficient Method for Reducing Reasoning Redundancy
• Entailed Opinion Matters: Improving the Fact-Checking Performance of Language Models by Relying on their Entailment Ability
• LAMP: Extracting Locally Linear Decision Surfaces from LLM World Models
• DSADF: Thinking Fast and Slow for Decision Making
• Critique Before Thinking: Mitigating Hallucination through Rationale-Augmented Instruction Tuning
• Learn to Think: Bootstrapping LLM Reasoning Capability Through Graph Representation Learning
• Understanding LLM Scientific Reasoning through Promptings and Model's Explanation on the Answers
• Reasoning Capabilities and Invariability of Large Language Models
• Do Automatic Comment Generation Techniques Fall Short? Exploring the Influence of Method Dependencies on Code Understanding
• RPO-RAG: Aligning Small LLMs with Relation-aware Preference Optimization for Knowledge Graph Question Answering
• A Survey of Reasoning with Foundation Models: Concepts, Methodologies, and Outlook
• Examining GPT's Capability to Generate and Map Course Concepts and Their Relationship
• Agentic Knowledgeable Self-awareness

Related

• Towards Reasoning in Large Language Models: A Survey
• Unlocking Structured Thinking in Language Models with Cognitive Prompting
• Prompting as Probing: Using Language Models for Knowledge Base Construction
• A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
• Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?