Reinforcement Learning in Partially Observable Markov Decision Processes using Hybrid Probabilistic Logic Programs

2010/11/27 by Emad Saad, Saad, Emad
Computer Science · #Artificial Intelligence (cs.AI) #Bayesian Modeling and Causal Inference #FOS: Computer and information sciences #Logic, Reasoning, and Knowledge #Multi-Agent Systems and Negotiation #Semantic Web and Ontologies #cs.AI

paper · pdf · doi:10.48550/arxiv.1011.5951

arxiv created 2010/11/27 · openalex publication_date 2010/11/27 · arxiv updated 2010/11/30 · openalex created_date 2022/09/01 · openalex updated_date 2026/07/28

Abstract

We present a probabilistic logic programming framework to reinforcement learning, by integrating reinforce-ment learning, in POMDP environments, with normal hybrid probabilistic logic programs with probabilistic answer set seman-tics, that is capable of representing domain-specific knowledge. We formally prove the correctness of our approach. We show that the complexity of finding a policy for a reinforcement learning problem in our approach is NP-complete. In addition, we show that any reinforcement learning problem can be encoded as a classical logic program with answer set semantics. We also show that a reinforcement learning problem can be encoded as a SAT problem. We present a new high level action description language that allows the factored representation of POMDP. Moreover, we modify the original model of POMDP so that it be able to distinguish between knowledge producing actions and actions that change the environment.

Citations

• Probabilistic Reasoning about Actions in Nonmonotonic Causal Theories
• Complexity of finite-horizon Markov decision process problems
• Planning and acting in partially observable stochastic domains

Related

• Provable Representation with Efficient Planning for Partial Observable Reinforcement Learning
• Markov Abstractions for PAC Reinforcement Learning in Non-Markov Decision Processes
• Reinforcement Learning with Temporal Logic Constraints for Partially-Observable Markov Decision Processes
• Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability
• Probabilistically Guaranteed Satisfaction of Temporal Logic Constraints During Reinforcement Learning