Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
2025/10/08 by Kawaharazuka, Kento, Oh, Jihoon, Yamada, Jun +2 · 16 citations
#Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotics (cs.RO)
paper · doi:10.48550/arxiv.2510.07077
Abstract
Amid growing efforts to leverage advances in large language models (LLMs) and vision-language models (VLMs) for robotics, Vision-Language-Action (VLA) models have recently gained significant attention. By unifying vision, language, and action data at scale, which have traditionally been studied separately, VLA models aim to learn policies that generalise across diverse tasks, objects, embodiments, and environments. This generalisation capability is expected to enable robots to solve novel downstream tasks with minimal or no additional task-specific data, facilitating more flexible and scalable real-world deployment. Unlike previous surveys that focus narrowly on action representations or high-level model architectures, this work offers a comprehensive, full-stack review, integrating both software and hardware components of VLA systems. In particular, this paper provides a systematic review of VLAs, covering their strategy and architectural transition, architectures and building blocks, modality-specific processing techniques, and learning paradigms. In addition, to support the deployment of VLAs in real-world robotic applications, we also review commonly used robot platforms, data collection strategies, publicly available datasets, data augmentation methods, and evaluation benchmarks. Throughout this comprehensive survey, this paper aims to offer practical guidance for the robotics community in applying VLAs to real-world robotic systems. All references categorized by training approach, evaluation method, modality, and dataset are available in the table on our project website: https://vla-survey.github.io .
Citations
- Grasp-MPC: Closed-Loop Visual Grasping via Value-Guided Model Predictive Control
- GR-3 Technical Report
- EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos
- A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation
- A Survey on Vision-Language-Action Models: An Action Tokenization Perspective
- Evo-0: Vision-Language-Action Model with Implicit Spatial Understanding
- WorldVLA: Towards Autoregressive Action World Model
- CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modeling
- RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
- RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies
- RLRC: Reinforcement Learning-based Recovery for Compressed Vision-Language-Action Models
- Steering Your Diffusion Policy with Latent Space Reinforcement Learning
- LeVERB: Humanoid Whole-Body Control with Latent Vision-Language Instruction
- RationalVLA: A Rational Vision-Language-Action Model with Dual System
- Time-Unified Diffusion Policy with Action Discrimination for Robotic Manipulation
- SAFE: Multitask Failure Detection for Vision-Language-Action Models
- BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Models
- BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulation
- Real-Time Execution of Action Chunking Flow Policies
- Fast ECoT: Efficient Embodied Chain-of-Thought via Thoughts Reuse
- SwitchVLA: Execution-Aware Task Switching for Vision-Language-Action Models
- SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
- OG-VLA: Orthographic Image Generation for 3D-Aware Vision-Language Action Model
- LoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon Embodied Tasks
- Agentic Robot: A Brain-Inspired Framework for Vision-Language-Action Models in Embodied Agents
- Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better
- TrackVLA: Embodied Visual Tracking in the Wild
- ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation
- DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation
- Hume: Introducing System-2 Thinking in Visual-Language-Action Model
- EgoZero: Robot Learning from Smart Glasses
- Is Single-View Mesh Reconstruction Ready for Robotics?
- FLARE: Robot Learning with Implicit World Modeling
- DreamGen: Unlocking Generalization in Robot Learning through Video World Models
- OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoning
- Unveiling the Potential of Vision-Language-Action Models with Open-Ended Multimodal Instructions
- VTLA: Vision-Tactile-Language-Action Model with Preference Learning for Insertion Manipulation
- Training Strategies for Efficient Embodied Reasoning
- UniSkill: Imitating Human Videos via Cross-Embodiment Skill Representations
- Pixel Motion as Universal Representation for Robot Control
- DexWild: Dexterous Human Interactions for In-the-Wild Robot Policies
- UniVLA: Learning to Act Anywhere with Task-centric Latent Actions
- Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challenges
- GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data
- OpenHelix: A Short Survey, Empirical Analysis, and Open-Source Dual-System VLA Model for Robotic Manipulation
- Interleave-VLA: Enhancing Robot Manipulation with Interleaved Image-Text Instructions
- ViSA-Flow: Accelerating Robot Skill Learning via Large-Scale Video Semantic Action Flow
- RoboGround: Robotic Manipulation with Grounded Vision-Language Priors
- NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks
- Gripper Keypose and Object Pointflow as Interfaces for Bimanual Robotic Manipulation
- π0.5: a Vision-Language-Action Model with Open-World Generalization
- A0: An Affordance-Aware Hierarchical Model for General Robotic Manipulation
- RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins
- Think Small, Act Big: Primitive Prompt Learning for Lifelong Robot Manipulation
- OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model
- MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulation
- ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generation
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
- Humanoid Policy ~ Human Policy
- VGGT: Visual Geometry Grounded Transformer
- LUMOS: Language-Conditioned Imitation Learning with World Models
- HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model
- CombatVLA: An Efficient Vision-Language-Action Model for Combat Tasks in 3D Action Role-Playing Games
- TLA: Tactile-Language-Action Model for Contact-Rich Manipulation
- FP3: A 3D Foundation Policy for Robotic Manipulation
- MoRE: Unlocking Scalability in Reinforcement Learning for Quadruped Vision-Language-Action Models
- VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation
- PointVLA: Injecting the 3D World into Vision-Language-Action Models
- iManip: Skill-Incremental Learning for Robotic Manipulation
- AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems
- Refined Policy Distillation: From VLA Generalists to RL Experts
- RaceVLA: VLA-based Racing Drone Navigation with Human-like Behaviour
- CognitiveDrone: A VLA Model and Evaluation Benchmark for Real-Time Cognitive Task Solving and Reasoning in UAVs
- DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Grasping
- Unified Video Action Model
- RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete
- Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
- ObjectVLA: End-to-End Open-World Object Manipulation Without Demonstration
- Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Models
- Humanoid-VLA: Towards Universal Humanoid Control with Visual Integration
- ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Model
- VLAS: Vision-Language-Action Model With Speech Instructions For Customized Robot Manipulation
- Qwen2.5-VL Technical Report
- SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation
- Pre-training Auto-regressive Robotic Models with 4D Representations
- GEVRM: Goal-Expressive Video Generation Model For Robust Visual Manipulation
- RoboBERT: An End-to-end Multimodal Robotic Manipulation Model
- DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control
- HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation
- ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy
- Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following
- HD-EPIC: A Highly-Detailed Egocentric Video Dataset
- SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
- VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Caching
- From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignment
- SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulation
- Integrating LMM Planners and 3D Skill Policies for Generalizable Manipulation
- UP-VLA: A Unified Understanding and Prediction Model for Embodied Agent
- Improving Vision-Language-Action Model with Online Reinforcement Learning
- SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model
- An Atomic Skill Library Construction Method for Data-Efficient Embodied Manipulation
- Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models
- SLIM: Sim-to-Real Legged Instructive Manipulation via Long-Horizon Visuomotor Learning
- Universal Actions for Enhanced Embodied Foundation Models
- FAST: Efficient Action Tokenization for Vision-Language-Action Models
- Shake-VLA: Vision-Language-Action Model-Based System for Bimanual Robotic Manipulations and Liquid Mixing
- UAV-VLA: Vision-Language-Action System for Large Scale Aerial Mission Generation
- Beyond Sight: Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Grounding
- Cosmos World Foundation Model Platform for Physical AI
- CoA-VLA: Improving Vision-Language-Action Models via Visual-Textual Chain-of-Affordance
- Qwen2.5 Technical Report
- RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation
- Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning
- TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policies
- RLDG: Robotic Generalist Policy Distillation via Reinforcement Learning
- RoboTron-Mani: All-in-One Multimodal Large Model for Robotic Manipulation
- ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Tasks
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
- NaVILA: Legged Robot Vision-Language-Action Model for Navigation
- Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videos
- Diffusion-VLA: Generalizable and Interpretable Robot Foundation Model via Self-Generated Reasoning
- SOLAMI: Social Vision-Language-Action Modeling for Immersive Interaction with 3D Autonomous Characters
- CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation
- HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos
- Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics
- RT-Affordance: Affordances are Versatile Intermediate Representations for Robot Manipulation
- DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution
- EgoMimic: Scaling Imitation Learning via Egocentric Video
- π0: A Vision-Language-Action Flow Model for General Robot Control
- Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning
- GPT-4o System Card
- Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Models
- A Dual Process VLA: Efficient Robotic Manipulation Leveraging VLM
- Diffusion Transformer Policy
- ALOHA Unleashed: A Simple Recipe for Robot Dexterity
- Latent Action Pretraining from Videos
- RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation
- GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation
- Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robust
- ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI
- RoboNurse-VLA: Robotic Scrub Nurse System based on Vision-Language-Action Model
- Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models
- ReVLA: Reverting Visual Domain Limitation of Robotic Foundation Models
- Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Models
- TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation
- Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments
- OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving
- In-Context Imitation Learning via Next-Token Prediction
- GR-MG: Leveraging Partially Annotated Data via Multi-Modal Goal-Conditioned Policy
- Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learning
- Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation
- Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model
- CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Driving
- Depth Helps: Improving Pre-trained RGB-based Policy with Depth Information Injection
- Actra: Optimized Transformer Architecture for Vision-Language-Action Models in Robot Learning
- The Llama 3 Herd of Models
- Qwen2 Technical Report
- HRP: Human Affordances for Robotic Pre-Training
- Affordance-Guided Reinforcement Learning via Visual Prompting
- Robotic Control via Embodied Chain-of-Thought Reasoning
- PaliGemma: A versatile 3B VLM for transfer
- Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphs
- RoboCAS: A Benchmark for Robotic Manipulation in Complex Object Arrangement Scenarios
- Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals
- Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning
- Open-TeleVision: Teleoperation with Immersive Active Visual Feedback
- LLaRA: Supercharging Robot Learning Data for Vision-Language Policy
- RoboUniView: Visual-Language Model with Unified View Representation for Robotic Manipulation
- Dreamitate: Real-World Visuomotor Policy Learning via Video Generation
- RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics
- OpenVLA: An Open-Source Vision-Language-Action Model
- RVT-2: Learning Precise Manipulation from Few Demonstrations
- BAKU: An Efficient Transformer for Multi-Task Policy Learning
- RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulation
- A Survey on Vision-Language-Action Models for Embodied AI
- Octo: An Open-Source Generalist Robot Policy
- Chameleon: Mixed-Modal Early-Fusion Foundation Models
- Evaluating Real-World Robot Manipulation Policies in Simulation
- Splat-MOVER: Multi-Stage, Open-Vocabulary Robotic Manipulation via Editable Gaussian Splatting
- Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation
- OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion
- DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
- Yell At Your Robot: Improving On-the-Fly from Language Corrections
- 3D-VLA: A 3D Vision-Language-Action Generative World Model
- Gemma: Open Models Based on Gemini Research and Technology
- DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation
- RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketches
- RT-H: Action Hierarchies Using Language
- Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation
- A Touch, Vision, and Language Dataset for Multimodal Alignment
- Aria Everyday Activities Dataset
- AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling
- 3D Diffuser Actor: Policy Diffusion with 3D Scene Representations
- Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots
- THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation
- Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models
- ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperation
- Bi-ACT: Bilateral Control-Based Imitation Learning via Action Chunking with Transformer
- SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning
- AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents
- General Flow as Foundation Affordance for Scalable Robot Learning
- Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data
- Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action
- Any-point Trajectory Modeling for Policy Learning
- QUAR-VLA: Vision-Language-Action Model for Quadruped Robots
- Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation
- Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis
- Foundation Models in Robotics: Applications, Challenges, and the Future
- VILA: On Pre-training for Visual Language Models
- SPOC: Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real World
- SARA-RT: Scaling up Robotics Transformers with Self-Adaptive Robust Attention
- Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
- On Bringing Robots Home
- Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
- An Embodied Generalist Agent in 3D World
- RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketches
- Vision-Language Foundation Models as Effective Robot Imitators
- RoboVQA: Multimodal Long-Horizon Reasoning for Robotics
- MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
- Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
- Putting the Object Back into Video Object Segmentation
- CCIL: Continuity-based Data Augmentation for Corrective Imitation Learning
- LoHoRavens: A Long-Horizon Language-Conditioned Benchmark for Robotic Tabletop Manipulation
- Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models
- Learning to Act from Actionless Videos through Dense Correspondences
- NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration
- Uni3D: Exploring Unified 3D Representation at Scale
- GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators
- Compositional Foundation Models for Hierarchical Planning
- Language Embedded Radiance Fields for Zero-Shot Task-Oriented Grasping
- BridgeData V2: A Dataset for Robot Learning at Scale
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- 3D Gaussian Splatting for Real-Time Radiance Field Rendering
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- CoTracker: It is Better to Track Together
- VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models
- LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning
- AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
- Safe & Accurate at Speed with Tendons: A Robot Arm for Exploring Dynamic Motion
- MotionGPT: Human Motion as a Foreign Language
- RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation
- LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
- SACSoN: Scalable Autonomous Control for Social Navigation
- Train Offline, Test Online: A Real Robot Learning Benchmark
- SoundStorm: Efficient Parallel Audio Generation
- DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objects
- Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
- Affordances from Human Videos as a Versatile Representation for Robotics
- Visual Instruction Tuning
- DINOv2: Learning Robust Visual Features without Supervision
- Micrograph segmentations for DDEVD
- Segment Anything
- Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
- Sigmoid Loss for Language Image Pre-Training
- EVA-CLIP: Improved Training Techniques for CLIP at Scale
- LERF: Language Embedded Radiance Fields
- GPT-4 Technical Report
- Annotated Point Clouds and Images from a Spatial-Semantic Perception Pipeline: Robotized Deconstruction at the Reference Construction Site, Aachen
- Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
- Diffusion policy: Visuomotor policy learning via action diffusion
- PaLM-E: An Embodied Multimodal Language Model
- Open-World Object Manipulation using Pre-trained Vision-Language Models
- ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth
- Scaling Robot Learning with Semantically Imagined Experience
- GenAug: Retargeting behaviors to unseen situations via Generative Augmentation
- ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills
- Efficient Online Reinforcement Learning with Offline Data
- Learning Universal Policies via Text-Guided Video Generation
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- Scalable Diffusion Models with Transformers
- Point-E: A System for Generating 3D Point Clouds from Complex Prompts
- RT-1: Robotics Transformer for Real-World Control at Scale
- Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting
- MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare
- CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learning
- Robust Speech Recognition via Large-Scale Weak Supervision
- Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models
- InstructPix2Pix: Learning to Follow Image Editing Instructions
- StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objects
- LAION-5B: An open large-scale dataset for training next generation image-text models
- Interactive Language: Talking to Robots in Real Time
- Flow Matching for Generative Modeling
- VIMA: General Robot Manipulation with Multimodal Prompts
- Grounding Language with Visual Affordances over Unstructured Data
- Latent Plans for Task-Agnostic Offline Reinforcement Learning
- Code as Policies: Language Model Programs for Embodied Control
- ProcTHOR: Large-Scale Embodied AI Using Procedural Generation
- PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies
- A Generalist Agent
- Flamingo: a Visual Language Model for Few-Shot Learning
- ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation
- GPT-NeoX-20B: An Open-Source Autoregressive Language Model
- Video Diffusion Models
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
- Egocentric Prediction of Action Target in 3D
- Detecting Twenty-thousand Classes using Image-level Supervision
- High-Resolution Image Synthesis with Latent Diffusion Models
- High-Resolution Image Synthesis with Latent Diffusion Models
- CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks
- GMFlow: Learning Optical Flow via Global Matching
- Masked Autoencoders Are Scalable Vision Learners
- RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learning
- LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs
- Ego4D: Around the World in 3,000 Hours of Egocentric Video
- Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets
- CLIPort: What and Where Pathways for Robotic Manipulation
- What Matters in Learning from Offline Human Demonstrations for Robot Manipulation
- ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations
- Habitat 2.0: Training Home Assistants to Rearrange their Habitat
- LoRA: Low-Rank Adaptation of Large Language Models
- Emerging Properties in Self-Supervised Vision Transformers
- Emerging Properties in Self-Supervised Vision Transformers
- Reset-Free Reinforcement Learning via Multi-Task Learning: Learning\n Dexterous Manipulation Behaviors without Human Intervention
- H2O: Two Hands Manipulating Objects for First Person Interaction\n Recognition
- MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale
- Rapid Exploration for Open-World Navigation with Latent Goal Models
- AST: Audio Spectrogram Transformer
- Learning Transferable Visual Models From Natural Language Supervision
- Taming Transformers for High-Resolution Image Synthesis
- PCT: Point cloud transformer
- Transporter Networks: Rearranging the Visual World for Robotic Manipulation
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Denoising Diffusion Implicit Models
- Multi-Stage Learning for Grasp-Constrained Object Manipulation with a Simulated Panda Robot
- Denoising Diffusion Probabilistic Models
- MediaPipe Hands: On-device Real-time Hand Tracking
- Understanding Human Hands in Contact at Internet Scale
- RoboTHOR: An Open Simulation-to-Real Embodied AI Platform
- RAFT: Recurrent All-Pairs Field Transforms for Optical Flow
- RAFT: Recurrent All-Pairs Field Transforms for Optical Flow
- SAPIEN: A SimulAted Part-based Interactive ENvironment
- Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
- RoboNet: Large-Scale Multi-Robot Learning
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
- RLBench: The Robot Learning Benchmark & Learning Environment
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
- A Short Note on the Kinetics-700 Human Action Dataset
- PyRep: Bringing V-REP to Deep Robot Learning
- HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million\n Narrated Video Clips
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
- Expressive Body Capture: 3D Hands, Face, and Body from a Single Image
- Habitat: A Platform for Embodied AI Research
- RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation
- Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data\n for Imitation
- SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing
- QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
- BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning
- BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning
- Universal Sentence Encoder
- PointCNN: Convolution On X-Transformed Points
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- AI2-THOR: An Interactive 3D Environment for Visual AI
- Neural Discrete Representation Learning
- FiLM: Visual Reasoning with a General Conditioning Layer
- Proximal Policy Optimization Algorithms
- The "something something" video database for learning and evaluating visual common sense
- Attention Is All You Need
- PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
- Mask R-CNN
- PointNet: Deep Learning on Point Sets for 3D Classification and\n Segmentation
- Deep Residual Learning for Image Recognition
- VQA: Visual Question Answering
- Microsoft COCO Captions: Data Collection and Evaluation Server
- ImageNet Large Scale Visual Recognition Challenge
- ImageNet Large Scale Visual Recognition Challenge
- Semi-Supervised Learning with Deep Generative Models
- Microsoft COCO: Common Objects in Context
- A Reduction of Imitation Learning and Structured Prediction to No-Regret\n Online Learning
- Long Short-Term Memory
- TriVLA: A Triple-System-Based Unified Vision-Language-Action Model with Episodic World Modeling for General Robot Control
- Tactile-VLA: Unlocking Vision-Language-Action Model's Physical Knowledge for Tactile Generalization
- Gemini Robotics: Bringing AI into the Physical World
- The Sound of Simulation: Learning Multimodal Sim-to-Real Robot Policies with Generative Audio
- RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot
Cited by
Related