Decision Making Agent Searching for Markov Models in Near-Deterministic\n World

2011/02/27 by Gabor Matuz, Matuz, Gabor, András Lörincz +1
Computer Science · #Artificial Intelligence in Games #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1102.5561

Abstract

Reinforcement learning has solid foundations, but becomes inefficient in\npartially observed (non-Markovian) environments. Thus, a learning agent -born\nwith a representation and a policy- might wish to investigate to what extent\nthe Markov property holds. We propose a learning architecture that utilizes\ncombinatorial policy optimization to overcome non-Markovity and to develop\nefficient behaviors, which are easy to inherit, tests the Markov property of\nthe behavioral states, and corrects against non-Markovity by running a\ndeterministic factored Finite State Model, which can be learned. We illustrate\nthe properties of architecture in the near deterministic Ms. Pac-Man game. We\nanalyze the architecture from the point of view of evolutionary, individual,\nand social learning.\n

Related

• Empirical Policy Optimization for n-Player Markov Games
• Markov Abstractions for PAC Reinforcement Learning in Non-Markov Decision Processes
• Reinforcement Learning in Non-Markovian Environments
• Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computation
• Policy Dispersion in Non-Markovian Environment