2012/07/04 by Scott Sanner, Craig Boutilier, Sanner, Scott +1 · 1 citation
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Elevator Systems and Control #FOS: Computer and information sciences #Reinforcement Learning in Robotics #Scheduling and Optimization Algorithms #cs.AI
paper · pdf · doi:10.48550/arxiv.1207.1415
Appears in Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence (UAI2005)
arxiv created 2012/07/04 · openalex publication_date 2012/07/04 · arxiv updated 2012/07/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We introduce a new approximate solution technique for first-order Markov decision processes (FOMDPs). Representing the value function linearly w.r.t. a set of first-order basis functions, we compute suitable weights by casting the corresponding optimization as a first-order linear program and show how off-the-shelf theorem prover and LP software can be effectively used. This technique allows one to solve FOMDPs independent of a specific domain instantiation; furthermore, it allows one to determine bounds on approximation error that apply equally to all domain instantiations. We apply this solution technique to the task of elevator scheduling with a rich feature space and multi-criteria additive reward, and demonstrate that it outperforms a number of intuitive, heuristicallyguided policies.