vix.ing · top · new · best · stats · spec

Optimal Continuous Time Markov Decisions

2015/07/10 by Butkova, Yuliya, Hatefi, Hassan, Hermanns, Holger +1
#FOS: Electrical engineering #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.1507.02876

Abstract

In the context of Markov decision processes running in continuous time, one of the most intriguing challenges is the efficient approximation of finite horizon reachability objectives. A multitude of sophisticated model checking algorithms have been proposed for this. However, no proper benchmarking has been performed thus far. This paper presents a novel and yet simple solution: an algorithm originally developed for a restricted subclass of models and a subclass of schedulers can be twisted so as to become competitive with the more sophisticated algorithms in full generality. As the second main contribution, we perform a comparative evaluation of the core algorithmic concepts on an extensive set of benchmarks varying over all key parameters: model size, amount of non-determinism, time horizon, and precision.

Related