2015/07/17 by Eugene A. Feinberg, Feinberg, Eugene A., Mark Edward Lewis +1 · 1 citation
Business, Management and Accounting · Computer Science · Engineering · #90B15 #90C40 #Advanced Queuing Theory Analysis #FOS: Mathematics #Optimization and Control (math.OC) #Optimization and Search Problems #Supply Chain and Inventory Management #Transportation and Mobility Innovations
paper · pdf · doi:10.48550/arxiv.1507.05125
openalex publication_date 2015/07/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This paper studies convergence properties of optimal values and actions for\ndiscounted and average-cost Markov Decision Processes (MDPs) with weakly\ncontinuous transition probabilities and applies these properties to the\nstochastic periodic-review inventory control problem with backorders, positive\nsetup costs, and convex holding/backordering costs. The following results are\nestablished for MDPs with possibly noncompact action sets and unbounded cost\nfunctions: (i) convergence of value iterations to optimal values for discounted\nproblems with possibly non-zero terminal costs, (ii) convergence of optimal\nfinite-horizon actions to optimal infinite-horizon actions for total discounted\ncosts, as the time horizon tends to infinity, and (iii) convergence of optimal\ndiscount-cost actions to optimal average-cost actions for infinite-horizon\nproblems, as the discount factor tends to 1.\n Being applied to the setup-cost inventory control problem, the general\nresults on MDPs imply the optimality of (s,S) policies and convergence\nproperties of optimal thresholds. In particular this paper analyzes the\nsetup-cost inventory control problem without two assumptions often used in the\nliterature: (a) the demand is either discrete or continuous or (b) the\nbackordering cost is higher than the cost of backordered inventory if the\namount of backordered inventory is large.\n