Linear programming for finite-horizon vector-valued Markov decision processes

2025/02/19 by Mifrani, Anas, Noll, Dominikus
#90C05 #90C29 #90C40 #FOS: Mathematics #Optimization and Control (math.OC)

paper · doi:10.48550/arxiv.2502.13697

Abstract

We propose a vector linear programming formulation for a non-stationary, finite-horizon Markov decision process with vector-valued rewards. Pareto efficient policies are shown to correspond to efficient solutions of the linear program, and vector linear programming theory allows us to fully characterize deterministic efficient policies. An algorithm for enumerating all efficient deterministic policies is presented then tested numerically in an engineering application.

Related

• Formally Verified Solution Methods for Infinite-Horizon Markov Decision Processes
• Finite-Horizon Markov Decision Processes with Sequentially-Observed Transitions
• Finite-Horizon Markov Decision Processes with State Constraints
• A Counterexample and a Corrective to the Vector Extension of the Bellman Equations of a Markov Decision Process
• Finite-Horizon Constrained MDPs With Both Additive And Multiplicative Utilities