A Deep Reinforcement Learning Framework for Continuous Intraday Market\n Bidding

2020/04/13 by Ioannis Boukas, Damien Ernst, Boukas, Ioannis +13 · 5 citations
Engineering · #Artificial Intelligence (cs.AI) #Electric Power System Optimization #Energy Load and Power Forecasting #FOS: Computer and information sciences #FOS: Economics and business #Machine Learning (cs.LG) #Smart Grid Energy Management #Trading and Market Microstructure (q-fin.TR)

paper · pdf · doi:10.48550/arxiv.2004.05940

openalex publication_date 2020/04/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The large integration of variable energy resources is expected to shift a\nlarge part of the energy exchanges closer to real-time, where more accurate\nforecasts are available. In this context, the short-term electricity markets\nand in particular the intraday market are considered a suitable trading floor\nfor these exchanges to occur. A key component for the successful renewable\nenergy sources integration is the usage of energy storage. In this paper, we\npropose a novel modelling framework for the strategic participation of energy\nstorage in the European continuous intraday market where exchanges occur\nthrough a centralized order book. The goal of the storage device operator is\nthe maximization of the profits received over the entire trading horizon, while\ntaking into account the operational constraints of the unit. The sequential\ndecision-making problem of trading in the intraday market is modelled as a\nMarkov Decision Process. An asynchronous distributed version of the fitted Q\niteration algorithm is chosen for solving this problem due to its sample\nefficiency. The large and variable number of the existing orders in the order\nbook motivates the use of high-level actions and an alternative state\nrepresentation. Historical data are used for the generation of a large number\nof artificial trajectories in order to address exploration issues during the\nlearning process. The resulting policy is back-tested and compared against a\nbenchmark strategy that is the current industrial standard. Results indicate\nthat the agent converges to a policy that achieves in average higher total\nrevenues than the benchmark strategy.\n

Cited by

• Joint Bidding on Intraday and Frequency Containment Reserve Markets
• Feature-driven reinforcement learning for photovoltaic in continuous intraday trading
• Gym-TORAX: Open-source software for integrating RL with plasma control simulators
• Model Predictive Control-Guided Reinforcement Learning for Implicit Balancing
• Bridging the Performance Gap Between Target-Free and Target-Based Reinforcement Learning

Related

• Reinforcement Learning for Bidding Strategy Optimization in Day-Ahead Energy Market
• Multi-market Energy Optimization with Renewables via Reinforcement Learning
• On-line reinforcement learning for optimization of real-life energy trading strategy
• A Machine Learning Approach for Prosumer Management in Intraday Electricity Markets
• Feature-driven reinforcement learning for photovoltaic in continuous intraday trading