2022/11/07 by Gianluca Fontanesi, Anding Zhu, Fontanesi, Gianluca +5 · 1 citation
Computer Science · Engineering · Environmental Science · #Distributed Control Multi-Agent Systems #FOS: Computer and information sciences #FOS: Electrical engineering #Robotics (cs.RO) #Signal Processing (eess.SP) #UAV Applications and Optimization #Wildlife-Road Interactions and Conservation #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2211.03696
openalex publication_date 2022/11/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The connectivity-aware path design is crucial in the effective deployment of autonomous Unmanned Aerial Vehicles (UAVs). Recently, Reinforcement Learning (RL) algorithms have become the popular approach to solving this type of complex problem, but RL algorithms suffer slow convergence. In this paper, we propose a Transfer Learning (TL) approach, where we use a teacher policy previously trained in an old domain to boost the path learning of the agent in the new domain. As the exploration processes and the training continue, the agent refines the path design in the new domain based on the subsequent interactions with the environment. We evaluate our approach considering an old domain at sub-6 GHz and a new domain at millimeter Wave (mmWave). The teacher path policy, previously trained at sub-6 GHz path, is the solution to a connectivity-aware path problem that we formulate as a constrained Markov Decision Process (CMDP). We employ a Lyapunov-based model-free Deep Q-Network (DQN) to solve the path design at sub-6 GHz that guarantees connectivity constraint satisfaction. We empirically demonstrate the effectiveness of our approach for different urban environment scenarios. The results demonstrate that our proposed approach is capable of reducing the training time considerably at mmWave.