Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policy

2025/08/18 by Tianyi Zhang, Zhang, Tianyi, Haonan Duan +9 · 4 citations
Arts and Humanities · Social Sciences · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Geography Education and Pedagogy #Linguistic Education and Pedagogy #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2508.13103

openalex publication_date 2025/08/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Vision-Language-Action (VLA) models frequently encounter challenges in generalizing to real-world environments due to inherent discrepancies between observation and action spaces. Although training data are collected from diverse camera perspectives, the models typically predict end-effector poses within the robot base coordinate frame, resulting in spatial inconsistencies. To mitigate this limitation, we introduce the Observation-Centric VLA (OC-VLA) framework, which grounds action predictions directly in the camera observation space. Leveraging the camera's extrinsic calibration matrix, OC-VLA transforms end-effector poses from the robot base coordinate system into the camera coordinate system, thereby unifying prediction targets across heterogeneous viewpoints. This lightweight, plug-and-play strategy ensures robust alignment between perception and action, substantially improving model resilience to camera viewpoint variations. The proposed approach is readily compatible with existing VLA architectures, requiring no substantial modifications. Comprehensive evaluations on both simulated and real-world robotic manipulation tasks demonstrate that OC-VLA accelerates convergence, enhances task success rates, and improves cross-view generalization. The code will be publicly available.

Citations

• Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy
• DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control
• SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model
• EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation
• Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation
• π₀: A Vision-Language-Action Flow Model for General Robot Control
• One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation
• WorldSimBench: Towards Video Generation Models as World Simulators
• The Ingredients for Robotic Diffusion Transformers
• RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation
• GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation
• Mamba Policy: Towards Efficient 3D Diffusion Policy with Hybrid Selective State Models
• GR-MG: Leveraging Partially Annotated Data via Multi-Modal Goal-Conditioned Policy
• Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals
• Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning
• Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion
• OpenVLA: An Open-Source Vision-Language-Action Model
• Octo: An Open-Source Generalist Robot Policy
• Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks
• DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
• 3D-VLA: A 3D Vision-Language-Action Generative World Model
• RT-H: Action Hierarchies Using Language
• 3D Diffuser Actor: Policy Diffusion with 3D Scene Representations
• ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation
• Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation
• SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution
• PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Play
• RobotGPT: Robot Manipulation Learning from ChatGPT
• Vision-Language Foundation Models as Effective Robot Imitators
• BridgeData V2: A Dataset for Robot Learning at Scale
• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
• Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition
• DINOv2: Learning Robust Visual Features without Supervision
• Goal-Conditioned Imitation Learning using Score-based Diffusion Policies
• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
• Diffusion policy: Visuomotor policy learning via action diffusion
• PaLM-E: An Embodied Multimodal Language Model
• LLaMA: Open and Efficient Foundation Language Models
• ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• Scalable Diffusion Models with Transformers
• RT-1: Robotics Transformer for Real-World Control at Scale
• Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks
• ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• CLIPort: What and Where Pathways for Robotic Manipulation
• ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations
• Diffusion Models Beat GANs on Image Synthesis
• Learning Transferable Visual Models From Natural Language Supervision
• Motion Planner Augmented Reinforcement Learning for Robot Manipulation in Obstructed Environments
• Denoising Diffusion Implicit Models
• ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation
• Denoising Diffusion Probabilistic Models
• Language Conditioned Imitation Learning over Unstructured Data
• A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms
• FiLM: Visual Reasoning with a General Conditioning Layer
• Attention Is All You Need
• Diffusion-VLA: Generalizable and Interpretable Robot Foundation Model via Self-Generated Reasoning
• RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot

Cited by

• Embodied Robot Manipulation in the Era of Foundation Models: Planning and Learning Perspectives
• Unify Robot Actions in Camera Frame
• VLA-LPAF: Lightweight Perspective-Adaptive Fusion for Vision-Language-Action to Enable More Unconstrained Robotic Manipulation
• From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model
• Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models

Related

• Clutter-Robust Vision-Language-Action Models through Object-Centric and Geometry Grounding
• cVLA: Towards Efficient Camera-Space VLAs
• From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model
• InSpire: Vision-Language-Action Models with Intrinsic Spatial Reasoning
• PosA-VLA: Enhancing Action Generation via Pose-Conditioned Anchor Attention