2020/01/07 by Keuntaek Lee, Lee, Keuntaek, Jason Gibson +3 · 4 citations
Computer Science · Engineering · Medicine · #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Glioma Diagnosis and Treatment #Machine Learning (cs.LG) #Robotic Path Planning Algorithms #Robotics (cs.RO) #Robotics and Sensor-Based Localization #Systems and Control (eess.SY) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2001.02307
openalex publication_date 2020/01/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Recently, vision-based control has gained traction by leveraging the power of\nmachine learning. In this work, we couple a model predictive control (MPC)\nframework to a visual pipeline. We introduce deep optical flow (DOF) dynamics,\nwhich is a combination of optical flow and robot dynamics. Using the DOF\ndynamics, MPC explicitly incorporates the predicted movement of relevant pixels\ninto the planned trajectory of a robot. Our implementation of DOF is\nmemory-efficient, data-efficient, and computationally cheap so that it can be\ncomputed in real-time for use in an MPC framework. The suggested Pixel Model\nPredictive Control (PixelMPC) algorithm controls the robot to accomplish a\nhigh-speed racing task while maintaining visibility of the important features\n(gates). This improves the reliability of vision-based estimators for\nlocalization and can eventually lead to safe autonomous flight. The proposed\nalgorithm is tested in a photorealistic simulation with a high-speed drone\nracing task.\n