Case Study: Testing Model Capabilities in Some Reasoning Tasks

2024/02/15 by Min Zhang, Zhang, Min, Takumi Sato +5
Computer Science · #AI-based Problem Solving and Planning #Bayesian Modeling and Causal Inference #Computation and Language (cs.CL) #FOS: Computer and information sciences #Semantic Web and Ontologies

paper · pdf · doi:10.48550/arxiv.2402.09967

openalex publication_date 2024/02/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large Language Models (LLMs) excel in generating personalized content and facilitating interactive dialogues, showcasing their remarkable aptitude for a myriad of applications. However, their capabilities in reasoning and providing explainable outputs, especially within the context of reasoning abilities, remain areas for improvement. In this study, we delve into the reasoning abilities of LLMs, highlighting the current challenges and limitations that hinder their effectiveness in complex reasoning scenarios.

Related

• Reasoning Capabilities and Invariability of Large Language Models
• Towards Reasoning in Large Language Models: A Survey
• Are LLMs the Master of All Trades? : Exploring Domain-Agnostic Reasoning Skills of LLMs
• Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
• Evaluating the Deductive Competence of Large Language Models