2021/12/22 by Julien Roy, Roger Girgis, Roy, Julien +7 · 2 citations
Computer Science · #Advanced Software Engineering Methodologies #FOS: Computer and information sciences #Machine Learning (cs.LG) #Software Engineering Research
paper · doi:10.48550/arxiv.2112.12228
openalex publication_date 2021/12/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29
The standard formulation of Reinforcement Learning lacks a practical way of specifying what are admissible and forbidden behaviors. Most often, practitioners go about the task of behavior specification by manually engineering the reward function, a counter-intuitive process that requires several iterations and is prone to reward hacking by the agent. In this work, we argue that constrained RL, which has almost exclusively been used for safe RL, also has the potential to significantly reduce the amount of work spent for reward specification in applied RL projects. To this end, we propose to specify behavioral preferences in the CMDP framework and to use Lagrangian methods to automatically weigh each of these behavioral constraints. Specifically, we investigate how CMDPs can be adapted to solve goal-based tasks while adhering to several constraints simultaneously. We evaluate this framework on a set of continuous control tasks relevant to the application of Reinforcement Learning for NPC design in video games.