Robotic Control via Embodied Chain-of-Thought Reasoning
2024/07/11 by Zawalski, Michał, Chen, William, Pertsch, Karl +3 · 103 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotics (cs.RO)
paper · doi:10.48550/arxiv.2407.08693
Abstract
A key limitation of learned robot control policies is their inability to generalize outside their training data. Recent works on vision-language-action models (VLAs) have shown that the use of large, internet pre-trained vision-language models as the backbone of learned robot policies can substantially improve their robustness and generalization ability. Yet, one of the most exciting capabilities of large vision-language models in other domains is their ability to reason iteratively through complex problems. Can that same capability be brought into robotics to allow policies to improve performance by reasoning about a given task before acting? Naive use of "chain-of-thought" (CoT) style prompting is significantly less effective with standard VLAs because of the relatively simple training examples that are available to them. Additionally, purely semantic reasoning about sub-tasks, as is common in regular CoT, is insufficient for robot policies that need to ground their reasoning in sensory observations and the robot state. To this end, we introduce Embodied Chain-of-Thought Reasoning (ECoT) for VLAs, in which we train VLAs to perform multiple steps of reasoning about plans, sub-tasks, motions, and visually grounded features like object bounding boxes and end effector positions, before predicting the robot action. We design a scalable pipeline for generating synthetic training data for ECoT on large robot datasets. We demonstrate, that ECoT increases the absolute success rate of OpenVLA, the current strongest open-source VLA policy, by 28% across challenging generalization tasks, without any additional robot training data. Additionally, ECoT makes it easier for humans to interpret a policy's failures and correct its behavior using natural language.
Cited by
- Embodied Robot Manipulation in the Era of Foundation Models: Planning and Learning Perspectives
- Clutter-Robust Vision-Language-Action Models through Object-Centric and Geometry Grounding
- Being-H0.7: A Latent World-Action Model from Egocentric Videos
- Unified Embodied VLM Reasoning with Robotic Action via Autoregressive Discretized Pre-training
- CoTinyVLA: Chain-of-Thought Distillation for a Sub-Billion-Parameter Vision-Language-Action Model
- Act, Think or Abstain: Complexity-Aware Adaptive Inference for Vision-Language-Action Models
- EVE: A Generator-Verifier System for Generative Policies
- MaP-AVR: A Meta-Action Planner for Agents Leveraging Vision Language Models and Retrieval-Augmented Generation
- PhysBrain: Human Egocentric Data as a Bridge from Vision Language Models to Physical Intelligence
- mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs
- Spatial-Aware VLA Pretraining through Visual-Physical Alignment from Human Videos
- D3D-VLP: Dynamic 3D Vision-Language-Planning Model for Embodied Grounding and Navigation
- An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges
- Openpi Comet: Competition Solution For 2025 BEHAVIOR Challenge
- Bridging Scale Discrepancies in Robotic Control via Language-Based Action Representations
- Mind to Hand: Purposeful Robotic Control via Embodied Reasoning
- SIMPACT: Simulation-Enabled Action Planning using Vision-Language Models
- Hierarchical Vision Language Action Model Using Success and Failure Demonstrations
- PosA-VLA: Enhancing Action Generation via Pose-Conditioned Anchor Attention
- CoT4AD: A Vision-Language-Action Model with Explicit Chain-of-Thought Reasoning for Autonomous Driving
- Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models
- MM-ACT: Learn from Multimodal Parallel Generation to Act
- DualVLA: Building a Generalizable Embodied Agent via Partial Decoupling of Reasoning and Action
- LocateAnything3D: Vision-Language 3D Detection with Chain-of-Sight
- Thinking in 360°: Humanoid Visual Search in the Wild
- MAPS: Preserving Vision-Language Representations via Module-Wise Proximity Scheduling for Better Vision-Language-Action Generalization
- Unifying Perception and Action: A Hybrid-Modality Pipeline with Implicit Visual Chain-of-Thought for Robotic Action Generation
- SPEAR-1: Scaling Beyond Robot Demonstrations via 3D Understanding
- Mantis: A Versatile Vision-Language-Action Model with Disentangled Visual Foresight
- Towards Deploying VLA without Fine-Tuning: Plug-and-Play Inference-Time VLA Policy Steering via Embodied Evolutionary Diffusion
- From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Models
- Scaling Graph Chain-of-Thought Reasoning: A Multi-Agent Framework with Efficient LLM Serving
- PixelVLA: Advancing Pixel-level Understanding in Vision-Language-Action Model
- RoboAfford++: A Generative AI-Enhanced Dataset for Multimodal Affordance Learning in Robotic Manipulation and Navigation
- Rethinking Progression of Memory State in Robotic Manipulation: An Object-Centric Perspective
- Using Language Models as Closed-Loop High-Level Planners for Robotics Applications: A Brief Overview and Benchmarks
- SlotVLA: Towards Modeling of Object-Relation Representations in Robotic Manipulation
- Embodiment Transfer Learning for Vision-Language-Action Models
- GUIDES: Guidance Using Instructor-Distilled Embeddings for Pre-trained Robot Policy Enhancement
- Maestro: Orchestrating Robotics Modules with Vision-Language Models for Zero-Shot Generalist Robots
- Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model
- DeepThinkVLA: Enhancing Reasoning Capability of Vision-Language-Action Models
- Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks
- Robotic Assistant: Completing Collaborative Tasks with Dexterous Vision-Language-Action Models
- SAM3D-Guided Object-Centric Representation Alignment for Vision-Language-Action Models
- CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decoding
- Do What You Say: Steering Vision-Language-Action Models via Runtime Reasoning-Action Alignment Verification
- BLM1: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning
- Learning Affordances at Inference-Time for Vision-Language-Action Models
- NeSyPr: Neurosymbolic Proceduralization For Efficient Embodied Reasoning
- Hierarchical DLO Routing with Reinforcement Learning and In-Context Vision-language Models
- VITA-E: Natural Embodied Interaction with Concurrent Seeing, Hearing, Speaking, and Acting
- MoTVLA: A Vision-Language-Action Model with Unified Fast-Slow Reasoning
- Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents
- Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Survey
- A Comprehensive Survey on World Models for Embodied AI
- InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policy
- RoVer: Robot Reward Model as Test-Time Verifier for Vision-Language-Action Model
- X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model
- IntentionVLA: Generalizable and Efficient Embodied Intention Reasoning for Human-Robot Interaction
- TrackVLA++: Unleashing Reasoning and Memory Capabilities in VLA Models for Embodied Visual Tracking
- Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
- FailSafe: Reasoning and Recovery from Failures in Vision-Language-Action Models
- FORGE-Tree: Diffusion-Forcing Tree Search for Long-Horizon Robot Manipulation
- EmbodiedCoder: Parameterized Embodied Mobile Manipulation via Modern Coding Model
- SITCOM: Scaling Inference-Time COMpute for VLAs
- Hybrid Training for Vision-Language-Action Models
- RoboPilot: Generalizable Dynamic Robotic Manipulation with Dual-thinking Modes
- From Code to Action: Hierarchical Learning of Diffusion-VLM Policies
- Preference-Based Long-Horizon Robotic Stacking with Multimodal Large Language Models
- Focusing on What Matters: Object-Agent-centric Tokenization for Vision Language Action models
- Generalizable Coarse-to-Fine Robot Manipulation via Language-Aligned 3D Keypoints
- Transferring Vision-Language-Action Models to Industry Applications: Architectures, Performance, and Challenges
- Pixel Motion Diffusion is What We Need for Robot Control
- Human-like Navigation in a World Built for Humans
- ImaginationPolicy: Towards Generalizable, Precise and Reliable End-to-End Policy for Robotic Manipulation
- Cross-Embodiment Transfer via Behavior-Aligned Representations
- RL2-VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models
- Pure Vision Language Action (VLA) Models: A Comprehensive Survey
- CollabVLA: Self-Reflective Vision-Language-Action Model Dreaming Together with Human
- Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challenges
- Igniting VLMs toward the Embodied Space
- OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft
- SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning
- Attribute-based Object Grounding and Robot Grasp Detection with Spatial Reasoning
- LLaDA-VLA: Vision Language Diffusion Action Models
- ANNIE: Be Careful of Your Robots
- OmniActor: A Generalist GUI and Embodied Agent for 2D&3D Worlds
- Robix: A Unified Model for Robot Interaction, Reasoning and Planning
- Uncertainty-Resilient Active Intention Recognition for Robotic Assistants
- Survey of Vision-Language-Action Models for Embodied Manipulation
- CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Models
- Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulation
- Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey
- Improving Pre-Trained Vision-Language-Action Policies with Model-Based Search
- Human Centric General Physical Intelligence for Agile Manufacturing Automation
- ReconVLA: Reconstructive Vision-Language-Action Model as Effective Robot Perceiver
- MolmoAct: Action Reasoning Models that can Reason in Space
- GraphCoT-VLA: A 3D Spatial-Aware Reasoning Vision-Language-Action Model for Robotic Manipulation with Ambiguous Instructions
- Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentation
- ADPro: a Test-time Adaptive Diffusion Policy via Manifold-constrained Denoising and Task-aware Initialization for Robotic Manipulation
- FedVLA: Federated Vision-Language-Action Learning with Dual Gating Mixture-of-Experts for Robotic Manipulation
- MapAgent: Trajectory-Constructed Memory-Augmented Planning for Mobile Task Automation
Related