Reinforcement Learning Problem Solving with Large Language Models

2024/04/29 by Sina Gholamian, Gholamian, Sina, Domingo Huh +1
Computer Science · #AI in Service Interactions #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Reinforcement Learning in Robotics #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2404.18638

openalex publication_date 2024/04/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large Language Models (LLMs) encapsulate an extensive amount of world knowledge, and this has enabled their application in various domains to improve the performance of a variety of Natural Language Processing (NLP) tasks. This has also facilitated a more accessible paradigm of conversation-based interactions between humans and AI systems to solve intended problems. However, one interesting avenue that shows untapped potential is the use of LLMs as Reinforcement Learning (RL) agents to enable conversational RL problem solving. Therefore, in this study, we explore the concept of formulating Markov Decision Process-based RL problems as LLM prompting tasks. We demonstrate how LLMs can be iteratively prompted to learn and optimize policies for specific RL tasks. In addition, we leverage the introduced prompting technique for episode simulation and Q-Learning, facilitated by LLMs. We then show the practicality of our approach through two detailed case studies for "Research Scientist" and "Legal Matter Intake" workflows.

Related

• On the Modeling Capabilities of Large Language Models for Sequential Decision Making
• LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models
• LLM-based Multi-Agent Reinforcement Learning: Current and Future Directions
• Integrating Large Language Models and Reinforcement Learning for Non-Linear Reasoning
• Large Language Model as a Policy Teacher for Training Reinforcement Learning Agents