2014/04/16 by Gavin Taylor, Taylor, Gavin, Connor Geer +3
Computer Science · Engineering · #Advanced Control Systems Optimization #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Reinforcement Learning in Robotics #Water resources management and optimization
paper · pdf · doi:10.48550/arxiv.1404.4258
openalex publication_date 2014/04/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Recent interest in the use of L1 regularization in the use of value function approximation includes Petrik et al.'s introduction of L1-Regularized Approximate Linear Programming (RALP). RALP is unique among L1-regularized approaches in that it approximates the optimal value function using off-policy samples. Additionally, it produces policies which outperform those of previous methods, such as LSPI. RALP's value function approximation quality is affected heavily by the choice of state-relevance weights in the objective function of the linear program, and by the distribution from which samples are drawn; however, there has been no discussion of these considerations in the previous literature. In this paper, we discuss and explain the effects of choices in the state-relevance weights and sampling distribution on approximation quality, using both theoretical and experimental illustrations. The results provide insight not only onto these effects, but also provide intuition into the types of MDPs which are especially well suited for approximation with RALP.