Optimal Decision-Making in Mixed-Agent Partially Observable Stochastic Environments via Reinforcement Learning

2019/01/04 by Roi Ceren, Ceren, Roi
Computer Science · Decision Sciences · Engineering · Mathematics · #Artificial Intelligence (cs.AI) #Artificial intelligence #Auction Theory and Applications #Computer science #Context (archaeology) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine learning #Markov chain #Markov decision process #Markov model #Markov process #Mathematics #Partially observable Markov decision process #Reinforcement Learning in Robotics #Reinforcement learning #Water resources management and optimization #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.1901.01325

Phd thesis, University of Georgia (2018)

arxiv created 2019/01/04 · openalex publication_date 2019/01/04 · arxiv updated 2019/01/08 · openalex created_date 2019/01/11 · openalex updated_date 2026/07/28

Abstract

Optimal decision making with limited or no information in stochastic environments where multiple agents interact is a challenging topic in the realm of artificial intelligence. Reinforcement learning (RL) is a popular approach for arriving at optimal strategies by predicating stimuli, such as the reward for following a strategy, on experience. RL is heavily explored in the single-agent context, but is a nascent concept in multiagent problems. To this end, I propose several principled model-free and partially model-based reinforcement learning approaches for several multiagent settings. In the realm of normative reinforcement learning, I introduce scalable extensions to Monte Carlo exploring starts for partially observable Markov Decision Processes (POMDP), dubbed MCES-P, where I expand the theory and algorithm to the multiagent setting. I first examine MCES-P with probably approximately correct (PAC) bounds in the context of multiagent setting, showing MCESP+PAC holds in the presence of other agents. I then propose a more sample-efficient methodology for antagonistic settings, MCESIP+PAC. For cooperative settings, I extend MCES-P to the Multiagent POMDP, dubbed MCESMP+PAC. I then explore the use of reinforcement learning as a methodology in searching for optima in realistic and latent model environments. First, I explore a parameterized Q-learning approach in modeling humans learning to reason in an uncertain, multiagent environment. Next, I propose an implementation of MCES-P, along with image segmentation, to create an adaptive team-based reinforcement learning technique to positively identify the presence of phenotypically-expressed water and pathogen stress in crop fields.

Citations

• The application of small unmanned aerial systems for precision agriculture: a review
• A Markovian Decision Process
• On the Shape of the Probability Weighting Function
• Plant Disease Detection by Imaging Sensors – Parallels and Specific Demands for Precision Agriculture and Plant Phenotyping
• Decisions from Experience and the Effect of Rare Events in Risky Choice
• The Complexity of Decentralized Control of Markov Decision Processes
• Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term
• Planning and acting in partially observable stochastic domains
• A Simplex Method for Function Minimization
• Behavioral Game Theory

Related

• Partially Observable Multi-Agent Reinforcement Learning with Information Sharing
• Partially Observable Mean Field Reinforcement Learning
• Policy Optimization in Multi-Agent Settings under Partially Observable Environments
• Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
• Multi-agent Reinforcement Learning: A Comprehensive Survey