2022/06/27 by Yuxiang Yang, Xiangyun Meng, Yang, Yuxiang +9 · 4 citations
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Artificial intelligence #Computer science #Computer vision #FOS: Computer and information sciences #Human Pose and Action Recognition #Human–computer interaction #Leverage (statistics) #Perception #Robot #Robotic Locomotion and Control #Robotics (cs.RO) #Robotics and Sensor-Based Localization #Semantics (computer science) #Simulation #Terrain #Traverse #cs.AI #cs.RO
paper · pdf · doi:10.48550/arxiv.2206.13631
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2022/06/27 · arxiv created 2022/10/10 · arxiv updated 2022/10/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The semantics of the environment, such as the terrain type and property, reveals important information for legged robots to adjust their behaviors. In this work, we present a framework that learns semantics-aware locomotion skills from perception for quadrupedal robots, such that the robot can traverse through complex offroad terrains with appropriate speeds and gaits using perception information. Due to the lack of high-fidelity outdoor simulation, our framework needs to be trained directly in the real world, which brings unique challenges in data efficiency and safety. To ensure sample efficiency, we pre-train the perception model with an off-road driving dataset. To avoid the risks of real-world policy exploration, we leverage human demonstration to train a speed policy that selects a desired forward speed from camera image. For maximum traversability, we pair the speed policy with a gait selector, which selects a robust locomotion gait for each forward speed. Using only 40 minutes of human demonstration data, our framework learns to adjust the speed and gait of the robot based on perceived terrain semantics, and enables the robot to walk over 6km without failure at close-to-optimal speed.