Learning to Fly -- a Gym Environment with PyBullet Physics for\n Reinforcement Learning of Multi-agent Quadcopter Control

2021/03/02 by Jacopo Panerati, Hehui Zheng, Panerati, Jacopo +9 · 12 citations
Computer Science · Engineering · #Reinforcement Learning in Robotics #Adversarial Robustness in Machine Learning #Traffic control and management

paper · pdf · doi:10.48550/arxiv.2103.02142

Abstract

Robotic simulators are crucial for academic research and education as well as\nthe development of safety-critical applications. Reinforcement learning\nenvironments -- simple simulations coupled with a problem specification in the\nform of a reward function -- are also important to standardize the development\n(and benchmarking) of learning algorithms. Yet, full-scale simulators typically\nlack portability and parallelizability. Vice versa, many reinforcement learning\nenvironments trade-off realism for high sample throughputs in toy-like\nproblems. While public data sets have greatly benefited deep learning and\ncomputer vision, we still lack the software tools to simultaneously develop --\nand fairly compare -- control theory and reinforcement learning approaches. In\nthis paper, we propose an open-source OpenAI Gym-like environment for multiple\nquadcopters based on the Bullet physics engine. Its multi-agent and vision\nbased reinforcement learning interfaces, as well as the support of realistic\ncollisions and aerodynamic effects, make it, to the best of our knowledge, a\nfirst of its kind. We demonstrate its use through several examples, either for\ncontrol (trajectory tracking with PID control, multi-robot flight with\ndownwash, etc.) or reinforcement learning (single and multi-agent stabilization\ntasks), hoping to inspire future research that combines control theory and\nmachine learning.\n

Cited by

• Dynamic Entropy Tuning in Reinforcement Learning Low-Level Quadcopter Control: Stochasticity vs Determinism
• Sim2Swim: Zero-Shot Velocity Control for Agile AUV Maneuvering in 3 Minutes
• Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges
• MagBotSim: Physics-Based Simulation and Reinforcement Learning Environments for Magnetic Robotics
• GO-Flock: Goal-Oriented Flocking in 3D Unknown Environments with Depth Maps
• A Simulation Evaluation Suite for Robust Adaptive Quadcopter Control
• ActivePose: Active 6D Object Pose Estimation and Tracking for Robotic Manipulation
• DiffAero: A GPU-Accelerated Differentiable Simulation Framework for Efficient Quadrotor Policy Learning
• DMPC-Swarm: Distributed Model Predictive Control on Nano UAV Swarms
• SwarmVLM: VLM-Guided Impedance Control for Autonomous Navigation of Heterogeneous Robots in Dynamic Warehousing
• Learning Agile Tensile Perching for Aerial Robots from Demonstrations
• ARMOR: Robust Reinforcement Learning-based Control for UAVs under Physical Attacks

Related

• Parallel Reinforcement Learning Simulation for Visual Quadrotor Navigation
• PyFlyt -- UAV Simulation Environments for Reinforcement Learning Research
• A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Platform
• Multitask Reinforcement Learning for Quadcopter Attitude Stabilization and Tracking using Graph Policy
• Deep Reinforcement Learning-based Quadcopter Controller: A Practical Approach and Experiments