Missing Velocity in Dynamic Obstacle Avoidance based on Deep Reinforcement Learning

2021/12/23 by Hart, Fabian, Waltz, Martin, Okhrin, Ostap
#FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2112.12465

Abstract

We introduce a novel approach to dynamic obstacle avoidance based on Deep Reinforcement Learning by defining a traffic type independent environment with variable complexity. Filling a gap in the current literature, we thoroughly investigate the effect of missing velocity information on an agent's performance in obstacle avoidance tasks. This is a crucial issue in practice since several sensors yield only positional information of objects or vehicles. We evaluate frequently-applied approaches in scenarios of partial observability, namely the incorporation of recurrency in the deep neural networks and simple frame-stacking. For our analysis, we rely on state-of-the-art model-free deep RL algorithms. The lack of velocity information is found to significantly impact the performance of an agent. Both approaches - recurrency and frame-stacking - cannot consistently replace missing velocity information in the observation space. However, in simplified scenarios, they can significantly boost performance and stabilize the overall training procedure.

Related

• Two-step dynamic obstacle avoidance
• Enhanced method for reinforcement learning based dynamic obstacle avoidance by assessment of collision risk
• Dynamically Feasible Deep Reinforcement Learning Policy for Robot Navigation in Dense Mobile Crowds
• Obstacle Avoidance using Dynamic Movement Primitives and Reinforcement Learning
• Dynamic Obstacle Avoidance through Uncertainty-Based Adaptive Planning with Diffusion