2020/10/15 by Juan Jose Garau Luis, Edward F. Crawley, Luis, Juan Jose Garau +3
Engineering · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Satellite Communication Systems #Space Satellite Systems and Control
paper · pdf · doi:10.48550/arxiv.2010.08015
openalex publication_date 2020/10/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The study and benchmarking of Deep Reinforcement Learning (DRL) models has\nbecome a trend in many industries, including aerospace engineering and\ncommunications. Recent studies in these fields propose these kinds of models to\naddress certain complex real-time decision-making problems in which classic\napproaches do not meet time requirements or fail to obtain optimal solutions.\nWhile the good performance of DRL models has been proved for specific use cases\nor scenarios, most studies do not discuss the compromises and generalizability\nof such models during real operations. In this paper we explore the tradeoffs\nof different elements of DRL models and how they might impact the final\nperformance. To that end, we choose the Frequency Plan Design (FPD) problem in\nthe context of multibeam satellite constellations as our use case and propose a\nDRL model to address it. We identify 6 different core elements that have a\nmajor effect in its performance: the policy, the policy optimizer, the state,\naction, and reward representations, and the training environment. We analyze\ndifferent alternatives for each of these elements and characterize their\neffect. We also use multiple environments to account for different scenarios in\nwhich we vary the dimensionality or make the environment nonstationary. Our\nfindings show that DRL is a potential method to address the FPD problem in real\noperations, especially because of its speed in decision-making. However, no\nsingle DRL model is able to outperform the rest in all scenarios, and the best\napproach for each of the 6 core elements depends on the features of the\noperation environment. While we agree on the potential of DRL to solve future\ncomplex problems in the aerospace industry, we also reflect on the importance\nof designing appropriate models and training procedures, understanding the\napplicability of such models, and reporting the main performance tradeoffs.\n