The Smoothed Complexity of Policy Iteration for Markov Decision Processes

2022/11/30 by Christ, Miranda, Yannakakis, Mihalis · 1 citation
#Computational Complexity (cs.CC) #Computer Science and Game Theory (cs.GT) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2212.00083

Abstract

We show subexponential lower bounds (i.e., 2^Ω(nc)) on the smoothed complexity of the classical Howard's Policy Iteration algorithm for Markov Decision Processes. The bounds hold for the total reward and the average reward criteria. The constructions are robust in the sense that the subexponential bound holds not only on the average for independent random perturbations of the MDP parameters (transition probabilities and rewards), but for all arbitrary perturbations within an inverse polynomial range. We show also an exponential lower bound on the worst-case complexity for the simple reachability objective.

Cited by

• Lower Bound on Howard Policy Iteration for Deterministic Markov Decision Processes

Related

• Improved and Generalized Upper Bounds on the Complexity of Policy Iteration
• Howard's Policy Iteration is Subexponential for Deterministic Markov Decision Problems with Rewards of Fixed Bit-size and Arbitrary Discount Factor
• On the Complexity of Policy Iteration
• Improved bound on the worst case complexity of Policy Iteration
• The Complexity of Markov Decision Processes