2022/09/20 by Guibin. Chen, Chen, Guibin., Xiaoying. Shi +1
Engineering · #Advanced Battery Technologies Research #Artificial Intelligence (cs.AI) #Electric Vehicles and Infrastructure #FOS: Computer and information sciences #Machine Learning (cs.LG) #Smart Grid Energy Management
paper · pdf · doi:10.48550/arxiv.2209.09772
openalex publication_date 2022/09/20 · openalex created_date 2022/09/22 · openalex updated_date 2026/07/28
This paper addresses the problem of optimizing charging/discharging schedules of electric vehicles (EVs) when participate in demand response (DR). As there exist uncertainties in EVs' remaining energy, arrival and departure time, and future electricity prices, it is quite difficult to make charging decisions to minimize charging cost while guarantee that the EV's battery state-of-the-charge (SOC) is within certain range. To handle with this dilemma, this paper formulates the EV charging scheduling problem as a constrained Markov decision process (CMDP). By synergistically combining the augmented Lagrangian method and soft actor critic algorithm, a novel safe off-policy reinforcement learning (RL) approach is proposed in this paper to solve the CMDP. The actor network is updated in a policy gradient manner with the Lagrangian value function. A double-critics network is adopted to synchronously estimate the action-value function to avoid overestimation bias. The proposed algorithm does not require strong convexity guarantee of examined problems and is sample efficient. Comprehensive numerical experiments with real-world electricity price demonstrate that our proposed algorithm can achieve high solution optimality and constraints compliance.