Reinforcement Learning in Partially Observable Markov Decision Processes\n using Hybrid Probabilistic Logic Programs

2010/11/27 by Emad Saad, Saad, Emad
Computer Science · #Artificial Intelligence (cs.AI) #Bayesian Modeling and Causal Inference #FOS: Computer and information sciences #Logic, Reasoning, and Knowledge #Multi-Agent Systems and Negotiation #Semantic Web and Ontologies

paper · pdf · doi:10.48550/arxiv.1011.5951

openalex publication_date 2010/11/27 · openalex created_date 2022/09/01 · openalex updated_date 2026/07/28

Abstract

We present a probabilistic logic programming framework to reinforcement\nlearning, by integrating reinforce-ment learning, in POMDP environments, with\nnormal hybrid probabilistic logic programs with probabilistic answer set\nseman-tics, that is capable of representing domain-specific knowledge. We\nformally prove the correctness of our approach. We show that the complexity of\nfinding a policy for a reinforcement learning problem in our approach is\nNP-complete. In addition, we show that any reinforcement learning problem can\nbe encoded as a classical logic program with answer set semantics. We also show\nthat a reinforcement learning problem can be encoded as a SAT problem. We\npresent a new high level action description language that allows the factored\nrepresentation of POMDP. Moreover, we modify the original model of POMDP so\nthat it be able to distinguish between knowledge producing actions and actions\nthat change the environment.\n

Citations

• Probabilistic Reasoning about Actions in Nonmonotonic Causal Theories
• Complexity of finite-horizon Markov decision process problems
• Planning and acting in partially observable stochastic domains

Related

• Provable Representation with Efficient Planning for Partial Observable Reinforcement Learning
• Markov Abstractions for PAC Reinforcement Learning in Non-Markov Decision Processes
• Reinforcement Learning with Temporal Logic Constraints for Partially-Observable Markov Decision Processes
• Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability
• Probabilistically Guaranteed Satisfaction of Temporal Logic Constraints During Reinforcement Learning