2024/05/25 by Michael J. Ahn, Montserrat Gonzalez Arenas, Ahn, Michael +47 · 2 citations
Computer Science · Engineering · #Anomaly Detection Techniques and Applications #FOS: Computer and information sciences #Industrial Vision Systems and Defect Detection #Robot Manipulation and Learning #Robotics (cs.RO)
paper · pdf · doi:10.48550/arxiv.2405.16021
openalex publication_date 2024/05/25 · openalex created_date 2024/05/29 · openalex updated_date 2026/07/28
Robots today can exploit the rich world knowledge of large language models to chain simple behavioral skills into long-horizon tasks. However, robots often get interrupted during long-horizon tasks due to primitive skill failures and dynamic environments. We propose VADER, a plan, execute, detect framework with seeking help as a new skill that enables robots to recover and complete long-horizon tasks with the help of humans or other robots. VADER leverages visual question answering (VQA) modules to detect visual affordances and recognize execution errors. It then generates prompts for a language model planner (LMP) which decides when to seek help from another robot or human to recover from errors in long-horizon task execution. We show the effectiveness of VADER with two long-horizon robotic tasks. Our pilot study showed that VADER is capable of performing complex long-horizon tasks by asking for help from another robot to clear a table. Our user study showed that VADER is capable of performing complex long-horizon tasks by asking for help from a human to clear a path. We gathered feedback from people (N=19) about the performance of the VADER performance vs. a robot that did not ask for help. https://google-vader.github.io/