2025/04/25 by Christel Baier, Baier, Christel, Krishnendu Chatterjee +5
Business, Management and Accounting · Engineering · Mathematics · #Advanced Queuing Theory Analysis #Advanced Wireless Network Optimization #FOS: Computer and information sciences #Logic in Computer Science (cs.LO) #Markov Chains and Monte Carlo Methods
paper · pdf · doi:10.48550/arxiv.2504.18277
openalex publication_date 2025/04/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This paper studies the expected value of multiplicative rewards, where rewards obtained in each step are multiplied (instead of the usual addition), in Markov chains (MCs) and Markov decision processes (MDPs). One of the key differences to additive rewards is that the expected value may diverge to infinity not only due to recurrent, but also due to transient states. For MCs, computing the value is shown to be possible in polynomial time given an oracle for the comparison of succinctly represented integers (CSRI), which is only known to be solvable in polynomial time subject to number-theoretic conjectures. Interestingly, distinguishing whether the value is infinite or 0 is at least as hard as CSRI, while determining if it is one of these two can be done in polynomial time. In MDPs, the optimal value can be computed in polynomial space. Further refined complexity results and results on the complexity of optimal schedulers are presented. The techniques developed for MDPs additionally allow to solve the multiplicative variant of the stochastic shortest path problem. Finally, for MCs and MDPs where an absorbing state is reached almost surely, all considered problems are solvable in polynomial time.