Managing engineering systems with large state and action spaces through deep reinforcement learning

2018/11/05 by C.P. Andriotis, Andriotis, C. P., Konstantinos G. Papakonstantinou +1 · 7 citations
Engineering · #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Multiagent Systems (cs.MA) #Safety Systems Engineering in Autonomy #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.1811.02052

openalex publication_date 2018/11/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Decision-making for engineering systems can be efficiently formulated as a Markov Decision Process (MDP) or a Partially Observable MDP (POMDP). Typical MDP and POMDP solution procedures utilize offline knowledge about the environment and provide detailed policies for relatively small systems with tractable state and action spaces. However, in large multi-component systems the sizes of these spaces easily explode, as system states and actions scale exponentially with the number of components, whereas environment dynamics are difficult to be described in explicit forms for the entire system and may only be accessible through numerical simulators. In this work, to address these issues, an integrated Deep Reinforcement Learning (DRL) framework is introduced. The Deep Centralized Multi-agent Actor Critic (DCMAC) is developed, an off-policy actor-critic DRL approach, providing efficient life-cycle policies for large multi-component systems operating in high-dimensional spaces. Apart from deep function approximations that parametrize large state spaces, DCMAC also adopts a factorized representation of the system actions, being able to designate individualized component- and subsystem-level decisions, while maintaining a centralized value function for the entire system. DCMAC compares well against Deep Q-Network (DQN) solutions and exact policies, where applicable, and outperforms optimized baselines that are based on time-based, condition-based and periodic policies.

Cited by

• Adaptive digital twins for predictive decision-making: Online Bayesian learning of transition dynamics
• A generative adversarial network optimization method for damage detection and digital twinning by deep AI fault learning: Z24 Bridge structural health monitoring benchmark validation
• A convolutional neural network deep learning method for model class selection
• Reinforcement Learning In Two Player Zero Sum Simultaneous Action Games
• AutoIndexer: A Reinforcement Learning-Enhanced Index Advisor Towards Scaling Workloads
• A Generative Physics-Informed Reinforcement Learning-Based Approach for Construction of Representative Drive Cycle
• Decoupled Hierarchical Reinforcement Learning with State Abstraction for Discrete Grids

Related

• Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints
• Multi-agent deep reinforcement learning with centralized training and decentralized execution for transportation infrastructure management
• Deep Reinforcement Learning: Framework, Applications, and Embedded Implementations
• Continuous Control for High-Dimensional State Spaces: An Interactive Learning Approach
• DeeP-Mod: Deep Dynamic Programming based Environment Modelling using Feature Extraction