Hilbert Space Embeddings of POMDPs

2012/10/16 by Yu Nishiyama, Nishiyama, Yu, Abdeslam Boularias +5 · 1 citation
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Network Security and Intrusion Detection #Water Systems and Optimization

paper · pdf · doi:10.48550/arxiv.1210.4887

openalex publication_date 2012/10/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes' rule to these distribution embeddings. Policies and value functions are defined on the feature space over states, which leads to a feature space expression for the Bellman equation. Value iteration may then be used to estimate the optimal value function and associated policy. Experimental results confirm that the correct policy is learned using the feature space representation.

Citations

• Modelling transition dynamics in MDPs with RKHS embeddings
• A Kernel Method for the Two-Sample Problem

Cited by

• Kernel Bayesian Inference with Posterior Regularization

Related

• Learning States Representations in POMDP
• Computationally Efficient PAC RL in POMDPs with Latent Determinism and Conditional Embeddings
• Learning in Observable POMDPs, without Computationally Intractable Oracles
• Modelling transition dynamics in MDPs with RKHS embeddings
• Scalable Policy-Based RL Algorithms for POMDPs