Stochastic Optimal Control with Side Information and Bayesian Learning

2026/02/25 by Johannes Milz, Alexander Shapiro, Enlu Zhou
#math.OC #math.ST #stat.TH

paper · pdf

Abstract

We study infinite-horizon stochastic optimal control problems with observable side information: a Markov chain that modulates an unknown context-conditional randomness distribution. Since this distribution is unknown, we propose a Bayesian reformulation based on a parametric density model and posterior predictive dynamics, which yields a Bayesian Bellman equation. We prove posterior consistency under Markov samples and, under correct specification and identifiability, uniform convergence of the Bayesian value function. Finally, we establish Bernstein--von Mises-type asymptotic normality for the data-driven contextual optimal value.

Citations

• Empirical PAC-Bayes Bounds for Markov Chains

Cited by

• Central Limit Theorems for Sample Average Approximations in Stochastic Optimal Control

Related

• Episodic Bayesian Optimal Control with Unknown Randomness Distributions
• Nonparametric Adaptive Bayesian Stochastic Control Under Model Uncertainty
• Optimal Control with Learning on the Fly: System with Unknown Drift
• Bayesian Learning of Optimal Policies in Markov Decision Processes with Countably Infinite State-Space
• The role of information in high dimensional stochastic optimal control